Sharepoint 2013 REST API: The C# Connection: Part 1 Using System.Net.Http.HttpClient   13 comments

This is the first of a multipart series on the Sharepoint REST API and will focus on using this API with C#.  At our shop we need to manipulate documents and list items between sites.  For our purposes we will be calling a custom WEB API HTTP Endpoint from Sharepoint workflows, these endpoints will then manipulatete Sharepoint objects using the Sharepoint REST API using C#.  If REST means nothing to you but a good sleep start here. The topics we will cover include:

  • HTTPClient Object (Framework 4.5)
  • Understanding and Manipulating the Result Sets returned by the REST API (Here)
  • Downloading and Uploading Documents from Document Libraries Using The REST API
  • Developing WEB API Endpoints To package our units of work (Web API 2.X)
  • Calling the WEB API Endpoints from Sharepoint Workflows using GET HTTP Web Service Action

I will not be looking at calling the REST API directly from within Sharepoint Workflows.  How many blog posts will this be?  I am not sure probably three posts in total.

Getting Our Feet Wet:  What an HTTP GET request to the Sharepoint REST API Looks like (10,000 foot view)

The new async HttpClient is a great advance over previous Microsoft Http clients.  You need to understand something about formal HTTP communications.  Specifically the roles of the different HTTP Verbs, Request Headers, HTTP Errors, and for POST commands, Response Headers and the Response Body.  So lets begin.  When we make AJAX calls in JavaScript all of our calls are passed through the machinery of the User Agent (Chrome, Firefox, etc.) and if you are using a helper library like JQuery, the helper library is doing a lot of the work for you.  When you switch to C# and the HTTPClient there is no User Agent and the client itself is quite thin so you have to a lot of work to get a well engineer solution.  So lets begin with some of the basics of an HTTP Get request.tumblr_ls7xrkHeuq1qchzcpo1_1280

There are four components of an HTTP Get request to be concerned about:

  • Target URL
    • What endpoint is the request headed for
  • Query String
    • What, if any, arguments are passed to the target
  • Accept Header
    • The format the results should be written back to the client
  • Authorization Header
    • What identity is the request being make under

For a Sharepoint REST API we want the site URL for the target URL and append to this \_api\.  So we start with something like:

http://myWeb.domain.com/_api/

In a non-RESTful interface the query string takes the form of:

?object=List&Title=MyList

In a RESTful interface this becomes appended to the base URL itself as something like

{baseURL}\List\ByTitle\MyList

Microsoft’s design for the REST API is a little eccentric and does not follow this pattern. A REST API call to get the meta-data for a list (not the list items) can look like this:

{myserver}/_api/web/lists/GetByTitle(‘MyList’)

where: {myserver}/_api/ is the baseURL and web/lists/GetByTitle(‘MyList’) are the arguments and values used in the call.

A GET call for a List, or any Sharepoint object returns a very complex objects whose structure is (basically) identical for different object types so once you understand the basic structure of the response it applies to many different objects.  We will not discuss that response object from a Sharepoint REST API Get request in this post. Note here that objects.  Anywhere the REST API allows an argument to be enclosed in single quote marks we may use an argument value with embedded spaces with out escaping the spaces.  Of concern to us at this point is the format of the response to the call. We have two choices XML format or JSON format.  When we are in the browser world and are using JavaScript to process the response, typically JSON is the preferred format, but XML can also be processed using JavaScript and assorted helper libraries.  When we are using a language like C# we can work most effectively with XML output.  In my case I prefer to receive XML and process that response using LINQ for XML.

Authentication for the Sharepoint REST API comes in three flavors:  OAuth token, Basic Authentication and (Windows) Integrated.  OAuth token only applies if you are calling from a Sharepoint APP (which means you are using JavaScript by definition) or from a Sharepoint Provider-Hosted High Trust App. (a very rare beast indeed and if you are using one you probably don’t need to be reading this post).  I will be concerned here with Basic and Integrated Authentication.

Now Lets Look at some Details Of A Sharepoint REST API call using C#

The HTTPClient object

Simple Anonymous HTTP GET call using HttpClient

        System.Net.Http.HttpClient _Client = new System.Net.Http.HttpClient();
_Client.BaseAddress = new Uri(baseURL);
HttpResponseMessage resp = client.GetAsync(uriString).Result;
string respString = resp.Content.ReadAsStringAsync().Result;
        if (resp.StatusCode != HttpStatusCode.OK)
{
throw new ApplicationException(“BAD”);
}

Where baseURL is a string representation of the site we are calling into for the REST API:

http://myServer.com/_api/

and

urrString is a string representation of the actual REST API call we which to make:

web/lists/GetByTitle(‘MyList’)

Since we are not setting an authorization method we are calling anonymously and this call in its current form would return a failure HTTP Status and the response body will contain information in the REST API default return format.  The HTTP status is returned as a Framework Enum System.Net.HttpStatusCode.  In formal terms HTTP status codes are numeric values.  If you want to recover the actual value you can cast HttpStatusCode to an int.  Thus  HttpStatusCode.OK cast as an integer is 200. This call as written wourld return HttpStatusCode.Unauthorized ( 401).

We need to associate an identity to the HttpClient object in order to get the REST API to return data to us.  I will discuss two ways to do this  (of the many options):

BASIC Authorization and Windows Integrated Authorization.  Basic Authorization depends on adding an Authorization Header to the client with  appropriate  user information (typically domain, user name and password) in a very specific format.

Header Name:  Authorization

Argument: BASIC domain\username\password

where domain\username\password are encoded as a Base 645 String:

static string _FormatBasicAuth(string domain, string user, string password)
{
const string format0 = @”{0}\{1}”;
const string format1 = @”{0}:{1}”;
string userName = string.Format(format0, domain, user);

return Convert.ToBase64String(Encoding.Default.GetBytes(

               string.Format(format1, userName, password)));

}

We then create the Authorization header on the HttpClient object:

System.Net.Http.HttpClient _Client = new System.Net.Http.HttpClient();

_Client.DefaultRequestHeaders.Authorization =

         new AuthenticationHeaderValue(“Basic”, _FormatBasicAuth(Domain, User, Password));

Note that the password is NOT encrypted so BASIC Authorization should only be used over a secure transport layer like SSL.

An alternative is to use Microsoft’s Integrated Authorization (also know as challenge and response).  This takes  network credentials (either of the active thread or created from initialization values) and allows the server to confirm the identity and validity of the caller without transmitting the data over a secure transport layer (‘that password it self is never passed over the wire’).  Here is an example using Network Credentials created on the fly:

_Client = new System.Net.Http.HttpClient(new HttpClientHandler()
{
new NetworkCredential(User, Password, Domain)
});

Alternatively the NetworkCredential object can be created from the running thread of your C# program (you do not need to supply the password  and other parts of the credentials explicitly in this case).  When set up this way you do not use an Authentication Header.  The Windows challenge and response pattern (Http Status 401) are controlled on the client side by the HttpClientHandler.

Assuming your REST API call is well formed and your credentials  are valid the response from the server will contain a representation of a Sharepoint entity (or entities).  I will discuss the structure of this data in a subsequent post.  You can control how the response is formatted by adding an Accept Header to the HttpClient Object.  For the REST API you have two choices: XML or JSON.  If you do not supply a header the default format is XML.  For the REST api both the XML and JSON take a very specific format in the Accept header.  XML requires a Accept Header (if supplied) with an argument of:

application/atom+xml

while JSON requires an argument of:

application/json;odata=verbose

The Accept Header for XML can be added to the HTTPClient as:

  _Client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue(@”application/atom+xml”));

Currently there is a bug in the HTTPClient which prevents adding an Accept Header for REST API formated JSON.  For JSON you need to use a different format:

_Client.DefaultRequestHeaders.Add(“Accept”,@”application/json;odata=verbose”);

Normally I use the XML format of the response for C# programs (although using JSON.NET you could process the return in JSON format).  Javascript coders will normally us the JSON format (although they could process the XML format).

My typical HTTP Get calls looks something like this:

System.Net.Http.HttpClient _Client = new System.Net.Http.HttpClient();
_Client.BaseAddress = new Uri(baseURL);
_Client.DefaultRequestHeaders.Accept.Add(

             new MediaTypeWithQualityHeaderValue(@”application/atom+xml”));

_Client.DefaultRequestHeaders.Authorization =

         new AuthenticationHeaderValue(“Basic”, _FormatBasicAuth(Domain, User, Password));

HttpResponseMessage resp = client.GetAsync(uriString).Result;
string respString = resp.Content.ReadAsStringAsync().Result;

if (resp.StatusCode==HttpStatusCode.OK){

        ProcessTheRESTAPIResults(respString)

}

switch (resp.StatusCode){

    //process Errors here

    //Note if the error is low level (you never reached the RESTAPI processor the

    //respString is a simple string message

    //if the error is returned by the RESTAPI processor the format is XML

}

OK, that is enough to get started, in the next post we will turn to more complex REST API GET calls and how to process the XML entities returned from successful Sharepoint REST API calls.

Sharepoint 2013 REST API: The C# Connection: Part 1 Using System.Net.Http.HttpClient

Sharepoint 2013 REST API: The C# Connection: Part 2 Query List or Item and Decoding The Meta-Data

Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data

Sharepoint 2013 REST API: The C# Connection: Part 4 Document Libraries, Folders And Files
Sharepoint 2013 REST API: The C# Connection: Part 5 REST API More on Folders and Other Odds & Ends

13 responses to “Sharepoint 2013 REST API: The C# Connection: Part 1 Using System.Net.Http.HttpClient

Subscribe to comments with RSS.

  1. Hi,
    I tried the following code :

    System.Net.Http.HttpClient _Client = new System.Net.Http.HttpClient();
    _Client.BaseAddress = new Uri(“https://test.sharepoint.com/”);
    _Client.DefaultRequestHeaders.Accept.Add(
    new MediaTypeWithQualityHeaderValue(@”application/atom+xml”));
    _Client.DefaultRequestHeaders.Authorization =
    new AuthenticationHeaderValue(“Basic”, _FormatBasicAuth(“https://test.sharepoint.com/”, “test@test.onmicrosoft.com”, “testpwd”));
    HttpResponseMessage resp = _Client.GetAsync(“https://test.sharepoint.com/_api/web/lists/”).Result;
    string respString = resp.Content.ReadAsStringAsync().Result;

    And this gives me the following error :
    System.UnauthorizedAccessException
    Access denied. You do not have permission to perform this action or access this resource.
    base {System.Net.Http.Headers.HttpHeaders} = {Transfer-Encoding: chunked
    X-SharePointHealthScore: 0
    X-Forms_Based_Auth_Required:URL

    I checked the credentials , they are valid.

    Any idea why it’s throwing error?

    • Your Authorization Header arguments are probably incorrect
      _Client.DefaultRequestHeaders.Authorization =
      new AuthenticationHeaderValue(“Basic”, _FormatBasicAuth(“https://test.sharepoint.com/”, “test@test.onmicrosoft.com”, “testpwd”));
      The first argument is your network domain name and “https://test.sharepoint.com/” is not it.
      Hope this helps.

  2. Hi,
    Thanks for the response.
    I am trying to connect to Sharepoint Online Premise(Cloud) to get documents in the “Document Library” by using the HTTPClient and the Sharepoint 2013 REST API .
    I tested the following rest service in the browser for this ,

    https://site_url/_vti_bin/ListData.svc/Documents

    which gives the MetaData of all the documents present in the Document Library of the particular web site.

    Now i am trying to do the same programmatically with your code.

    So in my case the credentials are valid.

    Any other thoughts?

    • We run into this. You were authenticated using your browser, no? Then you issued the get via the url line in the browser i presume. If you load up Fiddler and try this again I think you will see that the user agent is passing a fedauth cookie along with your request. In any case you need to be using your acs network domain in your auth header not the uri of your server.

    • In any case. As noted above you need to pass in the correct authentication information. In this case the identity you used to authenticate with the ACS in your cloud implementation (not your app domain).

  3. Pingback: Sharepoint 2013 REST API: The C# Connection: Part 2 Query List or Item and Decoding The Meta-Data | Cloud2013 Or Bust

  4. Pingback: Sharepoint 2013 REST API: The C# Connection: Part 3 Working With List Item Data | Cloud2013 Or Bust

  5. Pingback: Sharepoint 2013 REST API: The C# Connection: Part 4 Document Libraries, Folders And Files | Cloud2013 Or Bust

  6. Pingback: Sharepoint 2013 REST API: The C# Connection: Part 5 REST API More on Folders and Other Odds & Ends | Cloud2013 Or Bust

  7. Is there a clean method for calling the _api endpoint on an O365 cloud site? I’m about to try it.

    As Cloud2013 mentioned at the beginning, there are direct HTTP Web Service calls permitted within the 2013 workflows. But I tried this with my own Fiddler-generated FedAuth and rtFa cookie values, to no avail. The same simple test call I can successfully make in an authenticated browser – for example, siteURL/_api/web/lists – and also reproduce in Fiddler composer, fails authentication within the workflow, with all the same request headers.

    So now I’m gonna try this – will let y’all know soon

  8. Pingback: Sharepoint 2013 REST API call using C# | Stackforum.com

  9. Pingback: Access programmatically to SharePoint 2013 Online REST Web Services from Windows 8.1 Application | DL-UAT

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: