Call Sharepoint Online CSOM from an external application

There might be time when an external application want to talk to SharePoint Online (Office 365) without user interaction. Below are several scenarios that we might want to use SharePoint 2013 resources from external applications such as:

  • A console/windows application/service to perform administration to Skydrive Pro or SharePoint Online
  • Use SharePoint Online resources from Azure worker role i.e. uploading documents to a document library or adding items in a list or interact with the workflow,etc

These are steps to allow an external application to use site collection resources:

Firstly, we need to register a new SharePoint app. If the external application needs to access a site collection in SharePoint Online for example https://mytenant.sharepoint.com/sites/dev, we need to register an app by going to an application page called appregnew.aspx. For site collection above, it will be https://mytenant.sharepoint.com/sites/dev/_layouts/15/appregnew.aspx.

appregnew.aspx

  • Generate Client Id – and copy it to notepad
  • Generate Client Secret – and copy it to notepad
  • Title is your app title
  • App domain, the domain of your app, or anything such as guid. I normally put a URI that identifies the external app
  • Redirect URI, can be blank or put the current site collection url where this app registered.

Then click create button, it will register an Azure AD’s Service Principal with Id equals to the Client Id. This Service Principal will allow the OAuth process between the external application and SharePoint Online. (You can run this command from Office 365 Powershell console  to get more information about the service principal: Get-MsolServicePrincipal -AppPrincipalId <Client Id>) This msdn article provides some guideline about registering SharePoint app.

Next step, we need to set the permission for the app by going to /_layouts/15/appinv.aspx. The permission will authorize the external application to access SharePoint resources.
appinv

  • App Id: copy the Client Id we created at the first step and click Lookup. it will populate the other information except the Permission Request XML.
  • In the Permission Request XML put the below xml. The AllowAppOnlyPolicy flags that the registered app can be access by external application regardless the user.
  • The scope represents the permission right that the app can have. I noticed that with Write permission I am still getting access denied for uploading documents or adding items in a list. So I need to have FullControl permission.
<AppPermissionRequests AllowAppOnlyPolicy="true">
    <AppPermissionRequest Scope="http://sharepoint/content/sitecollection" Right="FullControl" />
</AppPermissionRequests>

Off course you need to be a site collection admin to be able to set permission as above, then click Create and click Trust It on the next screen. If you want to call SharePoint CSOM against any personal sites in SkyDrive Pro you need to register the app to have full control with tenant scope http://sharepoint/content/tenant. But only tenant admininstrator can register an app with this scope. This article contains all possible scopes that you can use in app permission.

TenantIDBefore we jump to the code we need to get the Realm (in the case of SharePoint Online, it is the Tenant Id). Go to /_layouts/15/appprincipals.aspx and copy the GUID after the ampersand to notepad. Click the image for more detail

Now we’re ready to write code, in your external application project add the TokenHelper.cs, you can get the file from any SharePoint App project then put the appsettings in the project’s app.config or web.config that will be used by the Token Helper as below

<appSettings>
    <add key="ClientId" value="the-client-id"/>
    <add key="ClientSecret" value="the-client-secret"/>
    <add key="Realm" value="the-tenant-id"/>
  </appSettings>

These setting will be used by the Token Helper to perform the OAuth process. Below is the code to get the SharePoint Client Context that can be used to access the site collection in SPO using CSOM.

 public static ClientContext GetClientContextForApp(Uri siteUrl)
 {
    var SharePointPrincipalId = "00000003-0000-0ff1-ce00-000000000000";
    var token = TokenHelper.GetAppOnlyAccessToken(SharePointPrincipalId, siteUrl.Authority, null).AccessToken;
    return TokenHelper.GetClientContextWithAccessToken(siteUrl.ToString(), token);
 }

The GetAppOnlyAccessToken’s 3rd parameter is targetrealm, it sets to null as it uses the one set in the appsettings. If you want to retrieve the realm dynamically you can call TokenHelper.GetRealmFromTargetUrl method, but this will make another https roundtrip to Azure AD.

With this approach we can use SharePoint Online resources from any external applications such as Azure Worker can upload files to a document library or insert items to a List or kick of a workflow in SharePoint.

PSConfig /SharePoint Configuration wizard runs longer when upgrading the servers.

During production farm (multi servers environment) SP1 upgrade,  I noticed that the psconfig was running significantly longer. I expected that the first run would be the only one that took very long time, but in my case the subsequent runs on the other servers took on average 25 minutes to finish. As we know that the first run will update the SharePoint databases so, it makes senses that it will take longer time in proportion to the number of the databases. But subsequent runs should only take around 5 minutes each.

I noticed that my config wizard seems paused at certain times, then I checked the upgrade log in the 14 hive\LOGS, which I found that during that apparent paused the log spit out alot of this below message, and they are a lot, as this site is in Mysite DB which has hundreds of site collections.

[PSCONFIG] [SPContentDatabaseSequence] [DEBUG] [3/3/2012 10:10:24 PM]: Site with Id = 961cfd1e-5cf8-4e40-8756-0032a517119b is contained in dictSitesNeedUpgrade, but is not added. Possible sitemap conflicts. Could not connect to http://localhost:32843/SecurityTokenServiceApplication/securitytoken.svc. TCP error code 10061: No connection could be made because the target machine actively refused it 127.0.0.1:32843.

Looking at that message, it seems that the psconfig tried to communicate with the Token Service, which obviously shut down during the process because the IIS service W3SVC (World Web Pulbhisng Service) is stopped.

So I looked at the Central Admin > Upgrade and Migration > Review Databases Status and I found that any content databases for any site collections that use Claim Based Authentication have Status field saying Database is up to date, but some sites are not completely upgraded. It became clear to me why the psconfig run much longer during the servers upgrade, because it tries to upgrade any claim based site collections by calling unavailable Security Token Services. It seems the psconfig try to call this service several times before it spits out the error message and it worsened in my case as there are hundreds of site collections of this type.

So my solution is before running the psconfig  to upgrade the servers (after successfully upgrade the database by running the psconfig at the first time), I run powershell commands to upgrade the databases that have status not completely upgraded. as mentioned in previous paragraph. This will work as the Token Service will be available. Run below powershell script for each db once, where the db_name is the ‘not fully upgraded’ database name. After run this command, my configuration wizard only runs for approximately 5 minutes in each server.

  $ct = Get-SPContentDatabase -Identity <db_name>
  Upgrade-SPContentDatabase -id  $ct.id