Configuring VENT integration of Azure Redis and Web App

To configure Azure Redis with the VNET support, we can follow the steps described in this document. And to integrate Azure web app with a VNET, there is a detailed document for it as well. In this post, I listed some of the common issues that one might hit during the configuration.

  1. The VNET integration for Azure Redis requires an empty subnet for the VNET that is created with Resource Manager. This subnet needs to be created before you create Azure Redis. Otherwise, the configuration would fail.
  2. The subnet for Azure Redis can be protected with a network security group (NSG). Usually the default NSG rules are good enough for protecting the connections. If you need further hardening, you will have to create rules based on the ports list in the Azure Redis document.
  3. To troubleshoot the connection between Azure web app and Azure Redis, you can use the Kudu of web app. There are two tools built in with the web app for network troubleshooting:
    nameresolver.exe can be used to test the DNS functionalities, and
    tcpping.exe
    can be used to test if the host and port can be pinged. But you cannot test the Redis function directly from the Kudu.
  4. Once the VNET integration is configured, the Redis console in Azure Portal will not work anymore. To test the Redis functions with tools such as redis-cli, you will have to build a VM in the VNET and connect to Azure Redis from it.
  5. If somehow your web app cannot access the Azure Redis, although the network configurations are correct, you can try to sync the network for App Service Plan. See this issue for details. Make sure you don’t hit any error when syncing the network.

Spring Boot, Azure Database for MySQL, and Azure App Service – Part 1

I recently played with Java and Azure App Service. What I was trying to find out is how the development experience would look like for Java developers if they want to build their applications with Azure App Service and Azure Database for MySQL.

There are some documents on Microsoft doc site, such as this one. It might be good enough for an experienced Java developer, but for someone like me who has limit Java experience, it is not easy to follow, and the sample is also too simple to make any sense for a real development. So I decided to try it myself and documented my experience here for others to reference. There would be a series of posts, and this is the first one. 

Prepare the dev environment

So instead of installing IntelliJ or Eclipse, I choose to use VSCode as my Java IDE. On my computer I’ve already had the VSCode installed. According to this tutorial, I just need to install JDK and Maven. I am a bit lost with the Java terms like Java SE, JDK, JRE and their versions, but I don’t want to be bothered. I choose to install OpenJDK because Oracle JDK requires a license. So here are steps to install OpenJDK. 

  1. Download OpenJDK from here. Windows version of OpenJDK is a zip file. Unzip it to C:\Program Files\Java so the root fold of the JDK would be something like C:\Program Files\Java\jdk-11.0.1
  2. Add an environment variable JAVA_HOME, set its value to the root of the JDK, for example, C:\Program Files\Java\jdk-11.0.1
  3. Add C:\Program Files\Java\jdk-11.0.1\bin to the system path. 
  4. With the above steps, OpenJDK is installed completely. To test if it works, open a command window and run java -version. It should print out the OpenJDK version and runtime information. 

When OpenJDK is installed, you can follow the vscode tutorial to download and install maven, and the Java Extension Pack for vscode. 

Create a MySQL database

Instead of installing MySQL on my local computer, I choose to create an Azure Database for MySQL instance as the dev database environment. It is easy to provision an Azure Database for MySQL instance. Azure has quick start for it. I also run the following SQL query to configure the database in Azure Cloud Shell. 

CREATE DATABASE tododb; -- Create a database
CREATE USER 'springuser'@'%' IDENTIFIED BY 'Spring1234'; -- Create a database user
GRANT ALL PRIVILEGES ON tododb.* TO 'springuser'@'%'; -- Grant user permissions to the database
FLUSH PRIVILEGES;

With the above preparation, we have a Java development environment and a MySQL database ready for the development. In the next post, I will start to create a Spring Boot REST API app with VSCode. Stay tuned. 

Crazyflie

Crazyflie 2.0
My Crazyflie 2.0

Azure Marketplace搞了一个叫做Super Human的推广活动,推广Azure Marketplace里的各种服务。这个活动推出了一些virtual labs,你如果正好对这些服务感兴趣,通过这些virtual labs,可以学习怎么在Azure中使用它们。

其实virtual labs不是我想说的重点。重点是,如果你成功做出了某个virtual lab的结果,会得到一个奖励。你可以选择得到一个为期3个月的Azure Pass,或者一个Crazyflie 2.0无人机。3个月的Azure Pass也许不错,可是我想大家应该都会选无人机吧?

所以,这个无人机才是我想说的重点,因为我做出了其中一个lab,并且拿到了无人机。这个活动的奖品只能寄到美国的地址,我辗转托了两个朋友才弄回新加坡。这个官网售价180美金的无人机,需要自己动手组装,上面的照片,就是组装好之后的样子,非常小巧。昨天装好之后,我拿去公园的空地上试了试,让无人机飞起来到没问题,只是操控还不是很熟悉,结果一个螺旋桨的塑料支架被我摔裂了。不过还好,包装里每个配件都提供多一个备用件,很好更换。

Remote Desktop for Linux VM on Azure

Usually you don’t need remote desktop or VNC on Linux servers running in the cloud. But as I wanted to try some scenarios with a Linux desktop and I actually don’t have a physical machine loaded with any Linux OS, I ended up setting up a Ubuntu server on Azure and enabling the remote desktop on it.

Obviously, I am not the first one who want to use remote desktop on servers running on cloud. There are plenty of posts on the internet talking about how to do it. Most of them are about using xrdp + xcfe4, including this one for Azure VMs. I am using Ubuntu 15.10 image. The only gotcha is that running the following command could uninstall the waagent service.

$sudo apt-get install ubuntu-desktop

This is a known issue that you can track on the github.com. To get waagent back, you have to reinstall it with the following:

$sudo apt-get install --reinstall walinuxagent

I ended up not installing the Ubuntu-desktop. Without it, you also avoid installing applications that you don’t need, such as those Office software.

Although xcfe4 is good enough as a lightweight window management system, I am more used to a GNOME like desktop environment. So I decided to try the MATE desktop. The configuration of it is very easy. Just run the following:

sudo apt-get install xrdp
sudo apt-get install mate-desktop-environment
echo mate-session >~/.xsession
sudo service xrdp restart

You may have to reboot the server after installing the desktop environment.

That’s all. I am using it now and so far so good.

The simplest way to copy blobs between different Azure storage accounts

Recently I have to move my Ubuntu VM to another Windows Azure subscription. In order to do it, I have to move the blobs of vhds to the storage of that subscription. At the beginning, I thought it was simple, just all I need would be copying the blobs to my local disk and then uploading them to the new storage account. But when I tried to do it, it appeared unachievable because the downloading and uploading would spend too much time that I cannot afford. So I searched on the web to find if there is any other better way to do it.

After searching on the web for a while, I think the simplest way I can figure out is to use AzCopy, a tool from Windows Azure Storage team. This tool leverages the Asynchronous Cross-Account Copy feature which was introduced in Windows Azure in June 2012. It is a very easy to use command line tool. For example, to copy all blobs in the source container to the destination container, you just need the following command.

AzCopy https://<sourceaccount>.blob.core.windows.net/<sourcecontainer>/ https://<destaccount>.blob.core.windows.net/<destcontainer>/  /sourcekey:<key> /destkey:<key> /S

The best part is all copying happens on Windows Azure server side. It is very fast. The speed could go up to more than 100MB/s. It really saves a lot time and effort comparing to the copying to local way. Give it a try if you need to copy or move blobs between different azure storage account.

配置Windows Azure上的Linux虚拟机

之前的post里讲过,我已经将这个blog从之前基于Windows Azure web role的解决方案,迁移到目前的部署在Windows Azure中的Linux虚拟机上了。这两个周末都在折腾这个虚拟机,记录一些细节。

首先,在Windows Azure中部署Linux虚拟机其实并不复杂,Windows Azure的文档里有相关的说明,照着做基本就可以创建好虚拟机,唯一需要注意的是ssh的证书。当我部署完成的时候,甚至惊讶,原来微软对开源系统的支持已经这么好了。

我使用的是Ubuntu 13.04的模版创建的虚拟机,一切顺利。虚拟机创建好之后,就可以像普通ubuntu机器一样来配置Apache, PHP和MySQL了,Ubuntu的help里有相关说明。拥有自己的虚拟机的好处是,我可以掌控上到系统下到应用的一切。目前我在这个虚拟机上运行了WordPress, tt-rss和tt-rss mobile。有空再折腾别的有趣的应用。

需要注意的几点:

  1. Windows Azure的虚拟机除了运行OS的.vhd之外,还需要单独挂一个.vhd,作为数据分区。你可能会注意到,创建好的虚拟机中,除了OS分区,还有一个分区/dev/sdb1,mount到/mnt/resource,而且空间挺大,有100多GB。但是这个分区和Azure里的Windows虚拟机的D:分区一样,是Azure虚拟机用来放临时数据的,它的数据不会persist到你自己的storage里,所以你还是需要自己创建一个.vhd挂上。只有在自己storage里的.vhd的数据才是persistent的。我之前将一个网站目录放在/mnt/resource里,昨天就碰到了404的错误,日志显示是I/O错误。
  2. Windows Azure的虚拟机目前没有备份或快照整个虚拟机的功能。文档里关于capture image的文章并不是备份的方法,capture image会unprovision整个虚拟机。官方的备份方法是,对storage里的.vhd做快照。当虚拟机有问题的时候,就恢复.vhd的快照,然后重建虚拟机。我的快照还没做,要找时间做一下。

Update: 一个可以用来给.vhd做快照的免费的Windows Azure storage explore工具:Azure Explorer

迁移到LAMP

虽然Blog现在不流行了,这个Blog我也不常更新,不过还是趁着新加坡国庆假期,把整个Blog迁移到了一个新的,基于Windows Azure IaaS的LAMP虚拟机上。这么做,一来是体验一下Azure虚拟机对开源系统的支持到底如何,二来也是解决系统稳定性和升级的问题,毕竟WordPress还是在原生的LAMP系统上更好一些。

之前用Azure的PaaS来构建WordPress系统,遇到的主要问题,就是系统升级。因为PaaS的特性,要升级WordPress的版本,需要重新部署应用。即使是安装个插件和主题之类的小部件,也需要把相应的部件部署到安装包里。这样不但麻烦,而且WordPress的在线升级和部件安装的功能完全无法使用。现在将它搬到LAMP上,这些问题都迎刃而解了。

初步体验,Windows Azure对开源系统的支持挺不错的。我从Gallery里直接选的Ubuntu Server 13.04,然后apt-get安装APH,配置apache,MySql和WordPress,一切顺利,顺带还把以前在DigiCert申请的SSL证书也用上了,目前看来一切正常。剩下的一个小问题是媒体文件的管理。我倾向于使用公开的媒体文件服务,比如flickr和YouTube,而不是把文件存储在WordPress里。不过,使用那些服务,上传和写文章就变得比较麻烦,不知道有没有好的编辑器插件可以解决这个上传的问题。或者试一下Windows Azure Storage for WordPress也说不定。

Blog修复记

本来以为blog放在Azure上可以一劳永逸,再也不会宕机了,谁知上周我在印度的时候,blog忽然出HTTP 500的错误,访问不了了。Azure在每个月底的时候,会重启服务器,重启的过程会重新部署package。我感觉可能是重新部署package时出错了。

我用remote desktop连接到一台Azure Server上,查看了startup tasks log。其中有一个错误,是在安装SQL Driver for PHP的时候出错了,这个driver的安装包无法从原来的地址下载。我试着用浏览器打开那个下载地址:http://download.microsoft.com/download/C/D/B/CDB0A3BB-600E-42ED-8D5E-E4630C905371/wpi_sqlsrv301_php53.EXE,果然不可用了。搜索了一下,原来随着WPI 4.0的发布,这个安装包的地址变了,新的地址是:http://download.microsoft.com/download/C/D/B/CDB0A3BB-600E-42ED-8D5E-E4630C905371/wpi40_sqlsrv301_php53.EXE。IIS的论坛里有完整的讨论:http://forums.iis.net/p/1189248/2031677.aspx

我update了一下部署Azure服务的package,现在blog终于回来了。更改package的步骤如下:

  1. 删除resources目录下原有的WebPICmdLine目录,这个是旧的WebPI预览版的命令行程序。
  2. 新建一个WebPICmd目录,将新的WebPICmd.exe复制到这个目录下,Microsoft.Web.PlatformInstaller.dll也需要复制过来。它们的默认安装路径是”C:Program FilesMicrosoftWeb Platform Installer”。
  3. 更新bin目录下的install-php-impl.cmd,将原来的命令行:

    “..resourcesWebPICmdLinewebpicmdline” /Products:PHP53,SQLDriverPHP53IIS,

    改成:

    “..resourcesWebPICmdwebpicmd” /Install /Products:PHP53,SQLDriverPHP53IIS,

    其它的保持不变。

  4. 重新package,然后选择更新已部署的服务。
  5. Package重新部署后,可能还需要reimage azure服务器。

希望这种更新程序后改变URL的情况不会频繁发生。

升级Windows Azure中的应用

昨天blog页面打开突然变成空的了,post无法显示,但是tags和comments列表是正常的。这说明不是数据库连接的问题。我连上Windows Azure的管理页面看了一下,两台WebRole的机器昨天被重启了。Windows Azure的机器重启之后,系统会被重置,应用会重新部署。我在前面的post里说过,WordPress for SQL的plug-in有一个bug,以前我是通过手工上传更新后的文件解决的,并没有更新安装包。应用重新部署后,就的文件又回来了,bug就又出现了。

为了一劳永逸,我昨天重新做了安装包,用升级的方式更新了应用的部署。Windows Azure支持增量更新和完全更新,我用的是增量更新的方式,因为我只需要更新一点点代码。步骤如下:

  1. 在Windows Azure管理界面,选择对WebRole升级,上传新的安装包和配置文件。因为我不需要升级配置,因此我将Azure中已有的配置拷贝了一份,作为新的配置文件。这能保证新配置和原配置一摸一样。
  2. 选择升级之后,两个node会被先后升级。升级完成后,可能会出现HTTP 500。我想主要是因为应用被重新部署了,但是机器没有重启,状态有混淆。
  3. 分别重启两个node之后,一切就恢复正常了。

完全更新的方式稍有不同,它需要新建一个staging部署,将新的安装包和配置文件部署在staging中,测试完成之后切换production和staging。这种方式适合有重大代码和配置更新的情况。

现在,似乎通过页面来发布新post会有问题,而用Windows Live Writer就没问题,我也就懒得管页面发布到底是什么问题了。Windows Live这个品牌不会继续了,Live Writer怎么办呢?

Now, Powered by Windows Azure

azure

早在年初的TechReady上,就看过Scott Guthrie演示在Windows Azure上跑WordPress。当时我正因为这个blog的主机商时不时掉线而伤脑筋,看了他的演示就想,不如将这个blog搬到Windows Azure上来。可是事情一直很多,直拖到现在。

Windows Azure的服务我在一两年前测试过,当时还只支持.NET。现在已经可以支持PHP,node.js,Java等等非微软系的技术了。对于在Windows Azure上部署WordPress,其实也很简单,下面这篇文档就包含了所有必要的信息:

How to deploy WordPress Multisite to Windows Azure using the WordPress scaffold

实际部署时,还是有些小细节需要注意:

  1. 这篇文档里的.phar下载的WordPress 3.2.1,不是最新的版本。可以将Scaffold原名从github上下载下来,修改相应的下载链接,来下载最新版的WordPress。
  2. 在Azure上部署的PHP程序,程序目录里的文件不是永久有效的。当Azure的虚机重启之后,会重新部署整个package。所以,在WordPress里下载theme和plugin不可行,需要先将需要的theme和plugin下载到项目的对应文件夹里,package在一起部署。
  3. 1.1.4版的数据库抽象库有一个小bug,在wp-contentmu-pluginswp-db-abstractiontranslationssqlsrvtranslations.php的第726行,正确代码是:$pattern = ”/LIMITs*(d+)((s*,?s*)(d+)*);{0,1}$/is”;
  4. 我在部署时遇到一个E:SiteRoot目录的写入问题。在倒入post或者在线安装plugin时,WordPress需要向这个目录写入临时文件,如果这个目录无法写入,则这些操作都无法做了。具体的原因我没找到,最终我是通过RDP到WebRole,手工修改目录权限解决的。这些权限会在下次VM重启之后被还原,还好我不是经常需要向这个目录写入文件。
  5. 另外,RDP的配置,可以参考这篇文章