If you would like to write .Net application that manages Azure HDInsight clusters you can use .Net client library. As a prerequisite step you need to get your subscriptionId and certificate to access it. You can get those by following instructions in How to install and configure Windows Azure PowerShell guide. Once this is done you can retrieve SubscripitonId and thumbprint of the certificate by using Get-AzureSubscription PowerShell command in Azure PowerShell Prompt. Next you can start creating your app.
  • First add HDInsight Cluster management NuGet package to your project:
    PM> Install-Package Microsoft.WindowsAzure.Management.HDInsight
  • Then get the certificate object from certificate store using thumbprint to identify it
            var store = new X509Store();
            store.Open(OpenFlags.ReadOnly);
            var cert = store.Certificates.Cast<X509Certificate2>().First(item => item.Thumbprint == "{Thumbrint of the certificate}");
  • Then you create HDInsightClient object using factory method
            var creds = new HDInsightCertificateCredential(new Guid("{Your subscription Id}"), cert);
            var client = HDInsightClient.Connect(creds);
  • Specify details of the cluster to create
            var clusterInfo = new ClusterCreateParameters()
            {
                Name = "{Your cluster name}",
                Location = "North Europe",
                DefaultStorageAccountName = "{Existing storage account}",
                DefaultStorageAccountKey = "{Storage account key}",
                DefaultStorageContainer = "{Existing storage container}",
                UserName = "{Cluster admin user name}",
                Password = "{Password for the cluster admin user}",
                ClusterSizeInNodes = {size of the cluster}
            };
  • Create cluster. CreateCluster function will poll automatically for the cluster status and will return when the cluster has finished provisioning. If you need method to return immediately you can use Async version.
            var clusterDetails = client.CreateCluster(clusterInfo);
  • To enumerate existing clusters use ListClusters function
            var clusters = client.ListClusters();
            foreach (var item in clusters)
            {
                Console.WriteLine("Cluster: {0}, Location: {1}", item.Name, item.Location);
            }
  • Remove cluster using DeleteCluster function
            client.DeleteCluster("{Name of the cluster}");

In addition to synchronous operations there is also asynchronous versions of the APIs: CreateClusterAsync, ListClustersAsync, DeleteclusterAsync.

Last edited Oct 25, 2013 at 12:29 AM by maxluk, version 2

Comments

rhizohm Aug 25, 2014 at 11:13 PM 
Note that the default account name needs to be NAME.blob.core.windows.net and same if you are adding a metastore, aka

HiveMetastore = new Metastore("SERVER_NAME.database.windows.net", "DBname", "USERNAME", "PASSWORD")