If you have a custom image in a Shared Image Gallery and you want to use it to create a pool in Azure Batch, this document, Use the Shared Image Gallery to create a custom pool, provides a pretty good guidance for it. I followed it to test the scenario and hit two minor issues.
As it is mentioned in another doc, AAD authentication is also a prerequisite for using shared image gallery. If you use --shared-key-auth with az batch account login, you would hit an anthentication error with Azure Cli. I raised an issue for the document and hopefully a note will be added to it.
There is no sample code to demonstrate how to create a pool with shared image gallery with Python.
So I wrote a simple sample in Python. It is based on the latest version (9.0.0) of Azure Batch package for Python. And it uses a service principal for the AAD authentication. The custom image I used for test was built on top of Ubuntu 18.04-LTS. So the node agent sku is ubuntu 18.04. It needs to be changed accordingly if other os version is used.
# Import the required modules from the
# Azure Batch Client Library for Python
import azure.batch._batch_service_client as batch
import azure.batch.models as batchmodels
from azure.common.credentials import ServicePrincipalCredentials
# Specify Batch account credentials
account = "<batch-account-name>"
batch_url = "<batch-account-url>"
ad_client_id = "<client id of the SP>"
ad_tenant = "<tenant id>"
ad_secret = "<secret of the SP>"
# Pool settings
pool_id = "LinuxNodesSamplePoolPython"
vm_size = "STANDARD_D2_V3"
node_count = 1
# Initialize the Batch client
creds = ServicePrincipalCredentials(
config = batch.BatchServiceClientConfiguration(creds, batch_url)
client = batch.BatchServiceClient(creds, batch_url)
# Create the unbound pool
new_pool = batchmodels.PoolAddParameter(id=pool_id, vm_size=vm_size)
new_pool.target_dedicated = node_count
# Configure the start task for the pool
start_task = batchmodels.StartTask(
start_task.run_elevated = True
new_pool.start_task = start_task
# Create an ImageReference which specifies the Marketplace
# virtual machine image to install on the nodes.
ir = batchmodels.ImageReference(
virtual_machine_image_id="<resource id of the image version in sig>"
# Create the VirtualMachineConfiguration, specifying
# the VM image reference and the Batch node agent to
# be installed on the node.
vmc = batchmodels.VirtualMachineConfiguration(
# Assign the virtual machine configuration to the pool
new_pool.virtual_machine_configuration = vmc
# Create pool in the Batch service
To configure Azure Redis with the VNET support, we can follow the steps described in this document. And to integrate Azure web app with a VNET, there is a detailed document for it as well. In this post, I listed some of the common issues that one might hit during the configuration.
The VNET integration for Azure Redis requires an empty subnet for the VNET that is created with Resource Manager. This subnet needs to be created before you create Azure Redis. Otherwise, the configuration would fail.
The subnet for Azure Redis can be protected with a network security group (NSG). Usually the default NSG rules are good enough for protecting the connections. If you need further hardening, you will have to create rules based on the ports list in the Azure Redis document.
To troubleshoot the connection between Azure web app and Azure Redis, you can use the Kudu of web app. There are two tools built in with the web app for network troubleshooting: nameresolver.exe can be used to test the DNS functionalities, and tcpping.execan be used to test if the host and port can be pinged. But you cannot test the Redis function directly from the Kudu.
Once the VNET integration is configured, the Redis console in Azure Portal will not work anymore. To test the Redis functions with tools such as redis-cli, you will have to build a VM in the VNET and connect to Azure Redis from it.
If somehow your web app cannot access the Azure Redis, although the network configurations are correct, you can try to sync the network for App Service Plan. See this issue for details. Make sure you don’t hit any error when syncing the network.
I recently played with Java and Azure App Service. What I was trying to find out is how the development experience would look like for Java developers if they want to build their applications with Azure App Service and Azure Database for MySQL.
There are some documents on Microsoft doc site, such as this one. It might be good enough for an experienced Java developer, but for someone like me who has limit Java experience, it is not easy to follow, and the sample is also too simple to make any sense for a real development. So I decided to try it myself and documented my experience here for others to reference. There would be a series of posts, and this is the first one.
Prepare the dev environment
So instead of installing IntelliJ or Eclipse, I choose to use VSCode as my Java IDE. On my computer I’ve already had the VSCode installed. According to this tutorial, I just need to install JDK and Maven. I am a bit lost with the Java terms like Java SE, JDK, JRE and their versions, but I don’t want to be bothered. I choose to install OpenJDK because Oracle JDK requires a license. So here are steps to install OpenJDK.
Download OpenJDK from here. Windows version of OpenJDK is a zip file. Unzip it to C:\Program Files\Java so the root fold of the JDK would be something like C:\Program Files\Java\jdk-11.0.1
Add an environment variable JAVA_HOME, set its value to the root of the JDK, for example, C:\Program Files\Java\jdk-11.0.1
Add C:\Program Files\Java\jdk-11.0.1\bin to the system path.
With the above steps, OpenJDK is installed completely. To test if it works, open a command window and run java -version. It should print out the OpenJDK version and runtime information.
When OpenJDK is installed, you can follow the vscode tutorial to download and install maven, and the Java Extension Pack for vscode.
Create a MySQL database
Instead of installing MySQL on my local computer, I choose to create an Azure Database for MySQL instance as the dev database environment. It is easy to provision an Azure Database for MySQL instance. Azure has quick start for it. I also run the following SQL query to configure the database in Azure Cloud Shell.
CREATE DATABASE tododb; -- Create a database
CREATE USER 'springuser'@'%' IDENTIFIED BY 'Spring1234'; -- Create a database user
GRANT ALL PRIVILEGES ON tododb.* TO 'springuser'@'%'; -- Grant user permissions to the database
With the above preparation, we have a Java development environment and a MySQL database ready for the development. In the next post, I will start to create a Spring Boot REST API app with VSCode. Stay tuned.
Usually you don’t need remote desktop or VNC on Linux servers running in the cloud. But as I wanted to try some scenarios with a Linux desktop and I actually don’t have a physical machine loaded with any Linux OS, I ended up setting up a Ubuntu server on Azure and enabling the remote desktop on it.
Obviously, I am not the first one who want to use remote desktop on servers running on cloud. There are plenty of posts on the internet talking about how to do it. Most of them are about using xrdp + xcfe4, including this one for Azure VMs. I am using Ubuntu 15.10 image. The only gotcha is that running the following command could uninstall the waagent service.
$sudo apt-get install ubuntu-desktop
This is a known issue that you can track on the github.com. To get waagent back, you have to reinstall it with the following:
$sudo apt-get install --reinstall walinuxagent
I ended up not installing the Ubuntu-desktop. Without it, you also avoid installing applications that you don’t need, such as those Office software.
Although xcfe4 is good enough as a lightweight window management system, I am more used to a GNOME like desktop environment. So I decided to try the MATE desktop. The configuration of it is very easy. Just run the following:
Recently I have to move my Ubuntu VM to another Windows Azure subscription. In order to do it, I have to move the blobs of vhds to the storage of that subscription. At the beginning, I thought it was simple, just all I need would be copying the blobs to my local disk and then uploading them to the new storage account. But when I tried to do it, it appeared unachievable because the downloading and uploading would spend too much time that I cannot afford. So I searched on the web to find if there is any other better way to do it.
After searching on the web for a while, I think the simplest way I can figure out is to use AzCopy, a tool from Windows Azure Storage team. This tool leverages the Asynchronous Cross-Account Copy feature which was introduced in Windows Azure in June 2012. It is a very easy to use command line tool. For example, to copy all blobs in the source container to the destination container, you just need the following command.
The best part is all copying happens on Windows Azure server side. It is very fast. The speed could go up to more than 100MB/s. It really saves a lot time and effort comparing to the copying to local way. Give it a try if you need to copy or move blobs between different azure storage account.
Windows Azure的虚拟机目前没有备份或快照整个虚拟机的功能。文档里关于capture image的文章并不是备份的方法，capture image会unprovision整个虚拟机。官方的备份方法是，对storage里的.vhd做快照。当虚拟机有问题的时候，就恢复.vhd的快照，然后重建虚拟机。我的快照还没做，要找时间做一下。
Update: 一个可以用来给.vhd做快照的免费的Windows Azure storage explore工具：Azure Explorer
初步体验，Windows Azure对开源系统的支持挺不错的。我从Gallery里直接选的Ubuntu Server 13.04，然后apt-get安装APH，配置apache，MySql和WordPress，一切顺利，顺带还把以前在DigiCert申请的SSL证书也用上了，目前看来一切正常。剩下的一个小问题是媒体文件的管理。我倾向于使用公开的媒体文件服务，比如flickr和YouTube，而不是把文件存储在WordPress里。不过，使用那些服务，上传和写文章就变得比较麻烦，不知道有没有好的编辑器插件可以解决这个上传的问题。或者试一下Windows Azure Storage for WordPress也说不定。
昨天blog页面打开突然变成空的了，post无法显示，但是tags和comments列表是正常的。这说明不是数据库连接的问题。我连上Windows Azure的管理页面看了一下，两台WebRole的机器昨天被重启了。Windows Azure的机器重启之后，系统会被重置，应用会重新部署。我在前面的post里说过，WordPress for SQL的plug-in有一个bug，以前我是通过手工上传更新后的文件解决的，并没有更新安装包。应用重新部署后，就的文件又回来了，bug就又出现了。