Running VS 2015 Tools for Docker in Azure VM

Recently I tried to deploy the ASP.NET 5 web app to remote docker host by following the post in this article. The reason I want to deploy to remote host because my development machine is an Azure VM and the docker machine from the Docker Toolbox  won’t be able to create a docker host (a Virtualbox VM) locally.

There are some missing information in the article, such as to be able to deploy to remote docker host (VM), the remote host needs to be registered to local docker machine. In this post I will explain how to register the remote docker host to docker machine running locally.

This is the steps based on this article with some additional steps

  1.  Prerequisite: The Create an Azure Docker Host VM instead of following the article I just using Azure portal to create a basic A0 VM based on Ubuntu 16.04 image and install docker by running wget -qO- https://get.docker.com/ | sh as explained in this article. In my case I have machine name: mydockertest.cloudapp.net and login name: mylogin
    And in addition, install Docker Toolbox for Windows. We need docker-machine and Git bash from this installation as VS Tools for Docker using it.
  2. Nothing changes in this step: Create ASP.NET 5 web app
  3. Add Docker support: nothing changes
  4. Point to remote docker host. Note. the {name-of-your-vm} is the docker host vm name you created in the step 1 above for example: mydockertest.cloudapp.net
  5. In this step we need to setup several endpoints (ports) in our remote docker host (ubuntu vm): 80, 2376 (for docker machine) and 22 (ssh). And then do the additional config below to register Ubuntu docker host to local docker-machine.

Below is the additional config to make the local docker-machine to talk to our remote host:

a).Allow port 2376 in the remote host (Ubuntu). SSH (using PuTTY) to remote Ubuntu host and run this command

sudo ufw allow 2376

b).Setting non-password sudo user. Still in the ssh session above run these commands.

mylogin@mydockertest.cloudapp.net:~$sudo adduser mylogin sudo
mylogin@mydockertest.cloudapp.net:~$sudo visudo
%sudo   ALL=(ALL) NOPASSWD:ALL

c). Setting ssh rsa key. Back to your development Windows VM, open Git bash and run below command. as in this article step 1.The command above will generate ssh keys and store it in /user/username/.ssh/.

   ssh-keygen

d). Then copy this public key to the remote host. Unfortunately step2 in the article above doesn’t work as Git bash in windows doesn’t have ssh-copy-id command. So we need to do the work around as in this article. Run below command in Git bash. Replace username,mylogin and mydockertest accordingly

  cat /users/username/.ssh/id_rsa.pub | ssh mylogin@mydockertest.cloudapp.net -p 22 "cat - >> ~/.ssh/authorized_keys"

e). Test is ssh is working by running from Git bash.

 ssh mydockertest.cloudapp.net 

f). After your development VM can ssh to remote host. We need to register the remote host to local docker machine by running this command below. Reference of this command is in here The location of the docker-machine.exe is in the Program Files where Docker Toolbox installed. Replace the mylogin and mydocktertest accordingly. You can remove the –debug flag from the command.

docker-machine.exe --debug create --driver generic --generic-ip-address mydockertest.cloudapp.net -generic-ssh-user mylogin mydockertest.cloudapp.net

g). Check if the the remote docker host is registered to local dockar machine by running this command

docker-machine.exe ls

docker-machine

Now we can build the Visual Studio Project as in the step 6 in this article. Please make sure you change the DockerMachineName in the Docker.props file as in the step 4 above. After the container is deployed you can ssh to the Ubuntu VM and run this command

sudo docker ps

docker-ps

SQL Azure Limit for SharePoint Autohosted App

To find the SQL Azure Edition and its maximum size in the SharePoint Online Autohosted app, firstly we need to get the connection string of the database used by the app by displaying it in an aspx page or more subtly returning it using Web API. Don’t forget that the connection string is defined in the AppSettings, snippet below shows how to get the value from the code.

public string GetConnectionString()
{
   return WebConfigurationManager.AppSettings["SqlAzureConnectionString"];
}

Normally you will get something like below.

Data Source=(a_server).database.windows.net;
Initial Catalog=db_(guid);
User ID=db_(guid_as_above)_dbo;
Password=(some_pwd)

Put this connection string to Visual Studio’s SQL explorer or SQL Management Studio. The User ID won’t have access to master db so that you need to connect directly to the database.
After it gets connected run this below scripts to get the SQL Edition and limit, thanks to Azure How-to talk series’ post

SELECT DATABASEPROPERTYEX('database name', 'EDITION')
SELECT CONVERT(BIGINT,DATABASEPROPERTYEX ( 'database name' , 'MAXSIZEINBYTES'))/1024/1024/1024 AS 'MAXSIZE IN GB'

And the result is Azure Web Edition and Max limit is 1 GB, and unfortunately we can’t change this using ALTER DATABASE command as shown here as we don’t have access to master db. Below is the screen shot from VS 2013′s SQL Server Explorer:

Azure_SQL_Autohosted

P.S. please let me know if there is a way to change the limit :)

Azure emulator wont start as CacheInstaller has stopped working

Several times, I am getting CacheInstaller error as shown below during the starting up of Azure emulator from Visual Studio. Most of the cases I can just ‘Close the program’ and the emulator will continue to start.
cacheInstaller
But there was one occasion that the CacheInstaller error dialog kept coming up each time I closed the dialog; and since then the emulator won’t start. So I clicked the ‘Debug the program’ to attach the CacheInstaller to Visual Studio. And copy the error message in the CacheInstallationException dialog (shown below) to notepad.
CacheInstallationException
And below is the message I got from the exception:

An unhandled exception of type 'Microsoft.ApplicationServer.Caching.AzureServerCommon.CacheInstallationException' occurred in CacheInstaller.exe
Additional information: Install Manifests : Script completed with error: ExitCode: 2
ErrorStream: Configuration error.
Configuration error.

ERROR: wevtutil.exe im Microsoft.ApplicationServer.EventDefinitions.man failed with error 15010
OutputStream: F:\Projects\My Cloud Project\My Cloud Project\csx\Debug\roles\MyCloudWorkerRole\plugins\Caching>C:\Windows\system32\unlodctr.exe /m:Microsoft.ApplicationServer.Caching.PerformanceCounter.man

Info: Successfully uninstalled the performance counters from the counter definition XML file Microsoft.ApplicationServer.Caching.PerformanceCounter.man.
F:\Projects\My Cloud Project\My Cloud Project\csx\Debug\roles\MyCloudWorkerRole\plugins\Caching>C:\Windows\system32\wevtutil.exe um Microsoft.WindowsFabric.EventDefinitions.man

Provider Microsoft-Windows-Fabric{{751c9dc0-4f51-44f6-920a-a620c7c2d13e}} is missing channels under the channelreferances registry key.
F:\Projects\My Cloud Project\My Cloud Project\csx\Debug\roles\MyCloudWorkerRole\plugins\Caching>C:\Windows\system32\wevtutil.exe um Microsoft.ApplicationServer.Caching.EventDefinitions.man
F:\Projects\My Cloud Project\My Cloud Project\csx\Debug\roles\MyCloudWorkerRole\plugins\Caching>C:\Windows\system32\wevtutil.exe um Microsoft.ApplicationServer.Caching.TracingEventDefinitions.man
F:\Projects\My Cloud Project\My Cloud Project\csx\Debug\roles\MyCloudWorkerRole\plugins\Caching>C:\Windows\system32\wevtutil.exe um Microsoft.ApplicationServer.EventDefinitions.man
F:\Projects\My Cloud Project\My Cloud Project\csx\Debug\roles\MyCloudWorkerRole\plugins\Caching>C:\Windows\system32\lodctr.exe /m:Microsoft.ApplicationServer.Caching.PerformanceCounter.man
Info: Successfully installed performance counters in F:\Projects\My Cloud Project\My Cloud Project\csx\Debug\roles\MyCloudWorkerRole\plugins\Caching\Microsoft.ApplicationServer.Caching.PerformanceCounter.man

F:\Projects\My Cloud Project\My Cloud Project\csx\Debug\roles\MyCloudWorkerRole\plugins\Caching>if 0 NEQ 0 (
echo.
 echo ERROR: lodctr.exe failed with error 0  1>&2
 exit /b 2
)

F:\Projects\My Cloud Project\My Cloud Project\csx\Debug\roles\MyCloudWorkerRole\plugins\Caching>C:\Windows\system32\wevtutil.exe in Microsoft.ApplicationServer.EventDefinitions.man
<b>Provider Microsoft-Windows-Fabric{{751c9dc0-4f51-44f6-920a-a620c7c2d13e}} is missing channels under the channelreferances registry key.</b>
F:\Projects\My Cloud Project\My Cloud Project\csx\Debug\roles\MyCloudWorkerRole\plugins\Caching>if 15010 NEQ 0 (
echo.
 echo ERROR: wevtutil.exe im Microsoft.ApplicationServer.EventDefinitions.man failed with error 15010  1>&2
 exit /b 2
)

From the highlighted message above, it is obvious that the wevtutil command is returning error (15010), instead of the expected value:0. And the error is something to do with some missing registry. Hmm, it looks like registry corruption and unfortunately I didn’t have restore point of my VM so that I could restore to the last working state. So I did more research on the wevtutil command, as for developer I am not really familiar with it. To cut the story short, the wevtutil command is provisioning/registering the custom event log defined in the EventDefinition.man file to the Event Viewer and it depends on the registry values in this locations: [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\WINEVT\Publisher]. Then I searched for registry key {751c9dc0-4f51-44f6-920a-a620c7c2d13e} under the Publisher node and find this node key: [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\WINEVT\Publishers\{751c9dc0-4f51-44f6-920a-a620c7c2d13e}] which is the event source for Azure Caching.

In the error message above it complains about missing channels. Under this registry node in my development VM, I only saw 2 channels (channel 1 and channel 2). So I did remote desktop to one of the worker roles in our UAT cloud services and check for this registry setting, and there were 3 channels (0,1, and 2). I added the channel 0 and their associated keys following the one in UAT and restart my visual studio. And finally I can debug my cloud project again.

Below is the registry settings that missing from my development VM:

CacheEventRegistry

Azure Service Bus Relay with WCF Routing Service

Azure Service Bus (SB) Relay provides us the ability create public endpoints for our internal on-premise/ corporate WCF services that resides behind proxy/firewall. This is quite useful during the development phase of any external apps that consume any on-premise LOB services. For example we can use our development cloud (MSDN) that in most cases don’t have Site to Site VPN setup to connect to these services. But if our app talk to more than 1 on premise services then each of them requires a SB Relay endpoint, so we need to manage multiple endpoint that point to the SB namespace. Luckily WCF 4.0 support routing service that enable message forwarding based on the different rules such as endpoint addresses. This diagram below shows the architecture:

SBRelayRoutingA WCF routing service host with  Azure Bus Relay endpoint maintains a routing table that maps the request URLs to on premise WCF service endpoints. I will outline the steps to set this up though it won’t be detail steps as most of them are covered in these MSDN articles (article_1 and article_2). So basically we need to combined the configuration from both articles.

  1. Setup Windows Azure Service Namespace as shown in this article_1. Let say that the service namespace is MyRouting
  2. Setup the Routing Service with Service Bus Endpoint: Create a Visual Studio’s Console Application or Windows Service, Add the Windows Azure Service Bus NuGet package (it will add the require dll and also add some WCF extensions that used by Service Bus Relay  in the app.config) and also System.ServiceModel.dll as well as System.ServiceModel.Routing.dll.
  3. In your console app’s program.cs (in windows service you add this routine in OnStart & OnStop methods) add the code below as shown in this article_2
     public static void Main(string[] args)
     {
        var host = new ServiceHost(typeof(RoutingService));
        host.Open();
        Console.WriteLine("host is listening");
        Console.ReadLine();
        host.Close();
     }
  4. Configuring the service host with SB endpoint and routing capability. Basically it combines the service host endpoint configuration as shown in this article 1 and routing xml as shown in the article 2. Which ending up likes below:
    <services>
          <service behaviorConfiguration="RoutingConfiguration"
                   name="System.ServiceModel.Routing.RoutingService">
            <endpoint binding="basicHttpRelayBinding"
                      contract="System.ServiceModel.Routing.IRequestReplyRouter"
                      address="https://myrouting.servicebus.windows.net/rttest"
                      behaviorConfiguration="sbTokenProvider"
                      bindingConfiguration="httpServiceBusBinding"/>
          </service>
    </services>

    The routing is defined in the service’s behavior configuration called Routing Configuration, it contains the ‘routing table’. As the endpoint is using the basicHttpRelayBinding the address is using https protocol.

  5. The behavior configuration as shown as below
     <behaviors>
          <serviceBehaviors>
            <behavior name="RoutingConfiguration">
              <routing filterTableName="LOBFilterTable" routeOnHeadersOnly="false" />
              <serviceDebug includeExceptionDetailInFaults="False"  />
            </behavior>
          </serviceBehaviors>
          <endpointBehaviors>
            <behavior name="sbTokenProvider">
              <transportClientEndpointBehavior>
                <tokenProvider>
                  <sharedSecret issuerName="owner" issuerSecret="**key**" />
                </tokenProvider>
              </transportClientEndpointBehavior>
            </behavior>
          </endpointBehaviors>
    </behaviors>
    <bindings>
           <basicHttpRelayBinding>
              <binding name="httpServiceBusBinding">
                <security mode="Transport" relayClientAuthenticationType="None" />
              </binding>
           </basicHttpRelayBinding>
    </bindings>

    The RoutingConfiguration defines the routing table with name LOBFilterTable and the Service Bus Relay endpoint is using sbTokenProvider endpoint’s behavior connect to the Azure Service Bus Relay. As my on-premise WCF services are using the basicHttpBinding, I use basicHttpRelayBinding for the Azure Service Bus endpoint’s binding.

  6. Next step is defining the routing table as shown below. The filterTable matches the filters with the client’s endpoint. The routing is based on EndpointAddress (this article shows different filter that we can use in WCF Routing Service), in this case the public endpoint on service bus is mapped to wcf client’s endpoint (on-premise WCF services endpoint).
     <routing>
          <filters>
            <filter name="LobSvc_A" filterType="EndpointAddress" filterData="https://myrouting.servicebus.windows.net/rttest/WcfEndpointA" />
            <filter name="LobSvc_B" filterType="EndpointAddress" filterData="https://myrouting.servicebus.windows.net/rttest/WcfEndpointB"/>
            <filter name="LobSvc_C" filterType="EndpointAddress" filterData="https://myrouting.servicebus.windows.net/rttest/WcfEndpointC"/>
          </filters>
          <filterTables>
            <filterTable name="LOBFilterTable">
              <add filterName="LobSvc_A" endpointName="LobSvc_A_Client" />
              <add filterName="LobSvc_B" endpointName="LobSvc_B_Client"/>
              <add filterName="LobSvc_B" endpointName="LobSvc_C_Client"/>
            </filterTable>
          </filterTables>
    </routing>
  7. Finally we put the client configuration as shown below. As you can see that the client endpoint’s names match with those in the filterTable
    <client>
      <endpoint address="your.onPremise.WCF.A.url" contract="*"
            binding="basicHttpBinding" name="LobSvc_A_Client" />
      <endpoint address="your.onPremise.WCF.B.url" contract="*"
            binding="basicHttpBinding" name="LobSvc_B_Client" />
      <endpoint address="your.onPremise.WCF.C.url" contract="*"
            binding="basicHttpBinding" name="LobSvc_C_Client" />
    </client>

    Of course you can add your own binding configuration and endpoint’s behavior to the client, such as connect using certain credentials. We have finished with the host so the next step is setting up the client connect this host.

  8. The client can be a web role in Azure or any external applications that need to connect to the on-premise WCF services. This code below show a generic method to create Service Bus Channel. the endpoint string parameter is the last part of endpoint in the filterdata filter’s configuration. In our example it can be WcfEndpointA, WcfEndpointB or WcfEndpointC, so WcfEndpointA will be routed to on-premise WCF A service.
     private static ChannelFactory<T> CreateChannelFactory<T>(string endPoint)
     {
       var binding = new BasicHttpRelayBinding();
       binding.Security.Mode = EndToEndBasicHttpSecurityMode.Transport;
       binding.Security.RelayClientAuthenticationType = RelayClientAuthenticationType.None;
                
       var serviceBusEndpoint = new EndpointAddress(ServiceBusEnvironment.CreateServiceUri("https", "myrouting", "rttest/" + endPoint));
       ChannelFactory<T> sbChannelFactory = new ChannelFactory<T>(binding, serviceBusEndpoint);
       sbChannelFactory.Endpoint.Behaviors.Add(
              new TransportClientEndpointBehavior { 
                 TokenProvider = TokenProvider.CreateSharedSecretTokenProvider("owner", "your sb secret key") 
       });
       return sbChannelFactory;
     }
  9. In conclusion we can have one Service Bus Endpoint that map to many on-premise WCF Services by utilizing WCF Routing Service capability.