submitting Pig Jobs

Sep 28, 2013 at 8:39 PM
Hi,

Can someone please confirm how can I submit pig jobs remotely using WebHCatHttpClient?

Thanks,
Coordinator
Sep 30, 2013 at 6:04 PM
Sep 30, 2013 at 8:26 PM
Thanks much..
Oct 1, 2013 at 11:55 PM
Hi,

Thanks for the URL, I tried the same steps to submit the pig job. I keep getting the error message as "Unable to connect to the remote server" if I run the app from my local machine.For HDinsight cluster do I have to change the port number?

httpClient = new WebHCatHttpClient(new Uri("http://clustername.azurehdinsight.net:50111"), "XXXXXXXXX", "XXXXXXXXX");
webhdfsClient = new WebHDFSClient(new Uri("http://clustername.azurehdinsight.net:50070"), "XXXXXXXXXX");

If I run the app from head node I keep getting error message as "Response Status Code does not indicate success:500 <Server Error">. When I am running it from headnode I am using server name as localhost.

httpClient = new WebHCatHttpClient(new Uri("http://localhost:50111"), "XXXXXXXXX", "XXXXXXXXX");
webhdfsClient = new WebHDFSClient(new Uri("http://localhost:50070"), "XXXXXXXXXX");

Any suggestions on how can I resolve this issue?
Coordinator
Oct 2, 2013 at 12:56 AM
Please use port 563 for job submission to the HDINSIGHT cluster remotely (From arbitrary client in the internet, not from head node). WebHDFS is not accessible remotely. Instead you can use Azure Blob Storage as file system. HDINSIGHT is integrated with it.
Oct 2, 2013 at 6:49 AM
Thanks much, Now I get a different error "A task was canceled." So looks like at least connectivity issue is resolved. But not sure what this error means. Please suggest