David Stringer at work

Software Development Articles

Using Atlassian Bamboo OnDemand for Visual Studio 2012 Projects – Part 3

Posted by David Stringer on 04/10/2012

This is the next part of a blog series detailing how to configure a Bamboo OnDemand build server for building Visual Studio 2012 projects. The previous part can be found here.

Phase 7 – Create the Amazon Image for the Bamboo OnDemand.

Follow these instructions to create our first base amazon image for use by Bamboo OnDemand.

  1. In the Amazon EC2 console stop the instance that we have been configuring.
  2. After the instance has stopped obtain the volume identifier of the root volume for this instance. To do this click on the sda1 link on the instance description lower pane and the volume identifier will be listed as the EBS ID on the pop up display describing the root volume.
  3. Go to the volumes pane and locate the volume with required identifier so that you can create a snapshot of the volume prior to running sysprep (the snapshot will take some time to perform as it is the first snapshot of this volume). You can track progess of creating the snapshot by selecting snapshots in the left of the console.
  4. Restart the instance once the snapshot has completed and then log back into the instance as the main Administrator account.
  5. Once the instance is restarted, launch the EC2ConfigService settings application, go to the Bundle tab and click the button labelled “Run Sysprep and Shutdown Now” and confirm you want to complete the sysprep process.
  6. Wait until the sysprep process has completed and the instance has been stopped.
  7. Now right click the instance and select create an image.
  8. In the Create Image dialog ensure the No Reboot option is selected and the Delete On Termination option is selected for the root volume /dev/sda1, then click the Create Image button.
  9. Now wait for the image to be created on the images – AMI pane of the Amazon Ec2 console.

When the image is ready you configure Bamboo OnDemand to start using it. In fact you can start this configuration process as soon as you know the AMI ID of the amazon image.

Phase 8 – Configure Bamboo OnDemand to use the custom elastic image.

In this phase we configuring Bamboo OnDemand to be aware of the amazon EC2 image you have just prepared.

Follow the instructions concerning the initial configuration of elastic bamboo refer to https://confluence.atlassian.com/display/BAMBOO041/Configuring+Elastic+Bamboo. Note that the Bamboo configuration is on the build tab of the Atlassian administration console. The Amazon AWS Access Key ID and AWS Secret Access Key must be configured in your Bamboo OnDemand account in order that Bamboo build server instances can be created. You can leave the Bamboo general configuration to start instances manually only whilst you are still testing whether your build server works. Once everything is working you can configure it to start and terminate instances automatically later on. We currently use the passive settings with a 300 second shutdown delay for normal CI building.

Follow the instructions concerning managing your elastic image configurations refer to https://confluence.atlassian.com/display/BAMBOO041/Managing+your+elastic+image+configurations in order to create a custom elastic image configuration. For our image the automatically attach EBS volumes option is not checked, the instance type is High-CPU Medium and product is Windows. Note that Atlassian can only support Bamboo elastic instances in the US-East region at present (this may change in the future).

Once the configuration is created you need to edit the capabilities of the image configuration to document that it is capable of building .Net 4.5 solutions. These are the capabilities you should add/modify:

  1. Add a capability of type Executable and sub type Command called MSTest 4.5 with the path C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\MSTest.exe.
  2. Add a capability of type Custom called DotNet 4.5 with the value true.
  3. Modify or replace capabilities of type JDK to that they are called JDK, JDK_1.7 and JDK_1.7_07 and all refer to the home directory c:\Program Files\Java\jdk1.7.0_07 (these entries match the JDK installed at the time we built the custom bamboo image). The default configuration provided by Bamboo will often reference outdated JDKs whose entries should be removed.

You should now be able to start a Bamboo Elastic Image using this configuration manually.

Once started you can monitor the progress of  initializing of the instance through Bamboo instance administration pages and also the Amazon Ec2 management console. This initialization could take up to 10 minutes as it requires two reboots during the instance configuration due to amazon renaming the machine following the initial windows rename. Also the first time the Bamboo Elastic Agent runs it has download a lot of java *.jar files which slows the startup even further.

To check the instance has started properly connect to the instance using RDP with the QuantivBuilder account and check that the file bamboo-elastic-agent.out in the Bamboo user profile directory correctly documents that the instance has started up and all the jar files have been downloaded.

Phase 9 – Configure a test build plan

We now need to test the build server actually works by creating a CI build plan for a sample Visual Studio 2012 solution. For our purposes we are using private source code repositories stored in GitHub under a company account.

Follow these instructions to create a CI build plan on the build tab of Atlassian Bamboo for a Visual Studio 2012 solution in a GitHub repository.

  1. Select Create a New Plan.
  2. Select the Jira project for which the plan is required.
  3. Specify the plan name, key and description.
  4. Select GitHub as the source repository and specify the a user name and password to access the GitHub repository.
  5. Now load the repositories and select the desired one and then select the required branch (usually the development branch in our case).
  6. Ensure ‘Use shallow clones’ options is selected for GitHub repositories.
  7. Leave the build strategy with default settings which specifies the plan is polling for repository changes every 180 seconds.
  8. Now start to add tasks to define the default job for the build plan. The source code checkout task from GitHub will have already been added.
  9. Add a MSBuild 4.0 (64-bit) Builder task for the specified solution. We tend to use command line options of the following form ‘/Property:Configuration=”Debug” /Property:Platform=”Any CPU” /target:Rebuild’ for a typical CI build. The working sub directory for the solution file should be specified so that you only need to specify the solution file name (with no preceding path) for the build project file.
  10. Add a script task to clean the old test results with an inline script body as follows:
     IF NOT EXIST TestResults\NUL GOTO NoTestResults
     rmdir /s /q TestResults
     :NoTestResults
    

    Don’t forget to specify the working sub directory for this script as the location of the solution file being built and tested.

  11. Add a command task to run the configured MSTest 4.5 task. We tend use command line options of the following form ‘/category:”CI&!Beta” /testcontainer:My.Assembly.Tests\bin\debug\My.Assembly.Tests.dll /testsettings:bamboo.testsettings’ for CI builds. Note that we do not use the preconfigured MSTest Runner task which only works for Visual Studio 2010 MSTest and cannot utilize testsettings files correctly. Again the working sub directory for this task is the location of the solution file.
  12. Finally add a MSTest parser task to collect the test results of the previous task using the results file SolutionSubDir\TestResults\Bamboo.trx.
  13. Now that the task for the default job are complete go to the Requirements page for the Job and add a new custom task requirement that refers to the custom capability DotNet 4.5 we added to the configured build server image earlier. The setting for the DotNet 4.5 capability can be simply set as exists (rather than equals or matches). This will ensure the test plan will use the correct build server image.
  14. The test plan can now be enabled on the main plan details page should be now ready to run.

For general instructions regarding build plans refer to https://confluence.atlassian.com/display/BAMBOO041/Using+Bamboo.

Note that our solutions have specific Bamboo.testsettings file available whose contents are something like the following:

<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="Bamboo" id="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Description>These are default test settings for a local test run.</Description>
  <NamingScheme baseName="Bamboo" appendTimeStamp="false" useDefault="false" />
  <Execution>
    <ExecutionThread apartmentState="MTA" />
    <TestTypeSpecific>
      <UnitTestRunConfig testTypeId="13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b">
        <AssemblyResolution>
          <TestDirectory useLoadContext="true" />
        </AssemblyResolution>
      </UnitTestRunConfig>
    </TestTypeSpecific>
    <AgentRule name="LocalMachineDefaultRole">
    </AgentRule>
  </Execution>
</TestSettings>

The above test settings file ensures the test results are written to a fixed location so that the bamboo build plan MSTest Results Parser task can locate the test results file. It also ensures threading tests run in a MTA apartment rather than STA apartment to improve the statistical accuracy of tests that use parallel processing.

Phase 10 – Optimizing or updating the custom elastic bamboo server image

This phase you will want to perform at least once to ensure the custom elastic bamboo server image is optimized so that downloads of NuGet packages and elastic bamboo agent java files are kept to a minimum.

To do this follow these instructions:

  1. Start an instance of the elastic agent you wish to optimize or update using the Atlassian Bamboo instance management screens.
  2. Wait for the instance to fully start (up to 10 mins). You can review the progress of the instance within the Atlassian Bamboo instance management screen.
  3. Once the bamboo instance is in the state idle, go to the Atlassian instance information screen for the instance (not the elastic agent information screen which details the capabilities of the agent on the instance), find the section titled “Remote Desktop Connection” and obtain the administrator password from the text below.
  4. Connect via RDP using the Administrator account with the password you obtained in the previous step.
  5. Go to the Amazon EC2 configuration console and right click the running instance so that you can change the instances shutdown behavior from terminate (the bamboo default) to stop.
  6. Force a build on the instance using a build plan if necessary so that NuGet package cache is correctly primed.
  7. Perform any new installations or updates to components on the build server as required.
  8. Go to services and stop the Elastic Bamboo Agent service. Also ensure any copies of Java.exe are killed following stopping the service.
  9. Delete the directory and all its contents C:\Users\Bamboo\bamboo-agent-home\xml-data\build-dir.
  10. Delete any log files in the directory C:\Users\Bamboo\bamboo-agent-home\logs.
  11. Delete the file C:\Users\Bamboo\bamboo-elastic-agent.out.
  12. Replace the contents of NuGet.config file in the C:\Users\DStringer\AppData\Roaming\NuGet directory with the basic following contents so that any encrypted passwords etc. are removed from the file (which will not work once the instance has been sysprepped):
    <?xml version="1.0" encoding="utf-8"?>
    <configuration>
      <packageRestore>
        <add key="enabled" value="True" />
      </packageRestore>
    </configuration>
    
  13. Now defragment the C: drive if necessary (you may need to do this for new build servers, remember to delete any files that you can from temporary directories before starting the defrag).
  14. If you have performed a defragment run the command C:\Program Files (x86)\SysInternals\sdelete -z C to zero out unused disk space (this is important for virtual disk frameworks such Amazon EBS).
  15. Launch the EC2ConfigService settings application, go to the Bundle tab and click the button labelled “Run Sysprep and Shutdown Now” and confirm you want to complete the sysprep process.
  16. Wait until the sysprep process has completed and the instance has been stopped.
  17. Now right click the instance and select create an image.
  18. In the Create Image dialog ensure the No Reboot option is selected and the Delete On Termination option is selected for the root volume /dev/sda1, then click the Create Image button.
  19. Now wait for the image to be created on the images pane of the Amazon Ec2 console.
  20. Now reconfigure your Bamboo elastic image configuration to use the updated image.
  21. Retest that new build server image is working by starting a new instance.

You can now reconfigure your Bamboo elastic image configuration to use the new updated Amazon Ec2 image and then test a build plan against this new image to ensure it still works. Don’t delete the old image until you are sure the new one works reliably.

Posted in Uncategorized | Leave a Comment »

Using Atlassian Bamboo OnDemand for Visual Studio 2012 Projects – Part 2

Posted by David Stringer on 04/10/2012

This is the next part of a blog series detailing how to configure a Bamboo OnDemand build server for building Visual Studio 2012 projects. The previous part can be found here.

Phase 4 – Installing a .Net 4.5 build and test environment.

Follow these instructions to make your build server capable of building .Net 4.5 based solutions developed in Visual Studio 2012.

  1. Connect to the instance using the Administrator account.
  2. Install Firefox latest version so that I can use Firefox as a download browser rather than Internet Explorer which is protected by IE Enhanced Security mode.
  3. Change Firefox options to allow you choose where you download files to.
  4. Create a directory called C:\Install. This is where I generally download all the installers that I use to customize the build server.
  5. Download the .Net 4.5 installer (dotNetFx45_x86_x64.exe) from http://www.microsoft.com/visualstudio/eng/downloads#net-45 to C:\Install.
  6. Now install .Net 4.5 framework.
  7. Download the SQL Server 2012 LocalDB 64-bit installer (LocalDB.msi) from http://www.microsoft.com/sqlserver/en/us/editions/2012-editions/express.aspx by selecting the 64-bit SQL Server Express LocalDB download option.
  8. Download the SQL Server 2012 Express Management tools 64-bit installer (SQLManagementStudio_x64_ENU.exe) from http://www.microsoft.com/sqlserver/en/us/editions/2012-editions/express.aspx by selecting the 64-bit SQL Server Management Studio Express download option.
  9. Now install both SQL 2012 LocalDB and the companion management tools afterwards. You may have to restart now to ensure these installations have completed.
  10. Download the Web Platform installer (wpilauncher.exe) from  http://www.microsoft.com/web/downloads/platform.aspx.
  11. Run the Web Platform Installer,  search for IIS 8.0 Express and install.
  12. Download the Windows 8 SDK installer (sdksetup.exe) from http://msdn.microsoft.com/en-US/windows/hardware/hh852363.
  13. Run the Windows 8 SDK Installer and select just the .Net Framework 4.5 Software Development Kit for installation (this will ensure the correct reference assemblies for .Net 4.5 will be placed on the build machine, something which happens by default when you install Visual Studio 2012).
  14. Copy the contents of the folder C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\WebApplications from a machine with Visual Studio 2012 installed to this build machine image at the same location (this is required if you need to build web projects which are tightly coupled to Visual Studio 2012 MSBuild target files).
  15. This next step is required to get MSTest 4.5 on the build machine which we require (If you are not using MSTest or are using an alternative such as NUnit then ignore this step). Download Agents for Visual Studio 2012 (x86) – DVD (English) (en_agents_for_visual_studio_2012_x86_dvd_921035.iso) from MSDN subscriber downloads (Quantiv can do this through our partner license).
  16. Within the iso file you will find a file called vstf_testagent.exe which can be used to install the test agents onto the build machine. You do not need to configure the test agents following installation as the Bamboo elastic agent will manage the automatic invocation of MSTest.exe.
  17. Copy the contents of SysinternalsSuite.zip obtained from http://technet.microsoft.com/en-us/sysinternals and unpack to C:\Program Files (x86)\Sysinternals for diagnostics use. As a follow up I tend to use the command ‘streams -d *.*’ in the C:\Program Files (x86)\Sysinternals directory to ensure all the downloaded files are unblocked.
  18. Download a copy of NuGet.exe from https://nuget.org/nuget.exe and place it in the directory C:\Program Files (x86)\NuGet which you may have to create.
  19. In the same C:\Program Files (x86)\NuGet directory create a file called StoreNuGetCredentials.cmd with contents similar to the following:
    “C:\Program Files (x86)\NuGet\Nuget.exe” sources Add -Name CompanyPrivateFeed -Source http://www.myget.org/F/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/ -UserName feedusername -Password feedpassword
    “C:\Program Files (x86)\NuGet\Nuget.exe” setApiKey xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx -Source http://www.myget.org/F/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/api/v2/package

    This file is used to ensure the correct credentials are stored in the %APPDATA%\NuGet\NuGet.config file for the Bamboo user profile so that builds can access private MyGet feeds for both installation and publishing (NuGet uses DPAPI to store credentials meaning that this file must be run for every new instance of a build server as it’s security identifiers will have be changed).

You could now check whether you can do a build and unit test of a sample .Net 4.5 project copied to the instance using the MSBuild and MSTest command line utilities.

Phase 5 – Installing and Configuring the Bamboo Elastic Agent

Follow these instructions to install and configure the Bamboo Elastic Agent. For general documentation regarding the creation of a custom elastic windows image refer to https://confluence.atlassian.com/display/BAMBOO041/Creating+a+custom+elastic+image+-+Windows (which we loosely follow).

  1. Download the Java SE JDK for both 32-bit and 64-bit versions from http://www.oracle.com/technetwork/java/javase/downloads/index-jsp-138363.html#javasejdk.
  2. Install both the 32-bit and 64-bit Java JDK’s and ensure that you have a system environment variable called JAVA_HOME that points to a directory such as C:\Program Files\Java\jdk1.7.0_07 (in this example we have installed JDK V1.7.0 update 7).
  3. Download Git for windows (msysgit) from http://code.google.com/p/msysgit/downloads/list?q=full+installer+official+git. We chose to download Git-1.7.11-preview20120710.exe at the time of this blog post.
  4. Install Git for Windows taking the default options apart from not installing Quick Launch and Desktop shortcuts and ensuring the ‘commit line endings as is’ option is selected as we are in a pure windows development environment. Git requires no further configuration as the bamboo elastic agent will automate it passing the correct credentials to access the required repositories.
  5. Download the Bamboo Elastic Agent from https://maven.atlassian.com/content/repositories/atlassian-public/com/atlassian/bamboo/atlassian-bamboo-elastic-image. We downloaded version 2.2 at the time of this blog which matched the Bamboo OnDemand server version 4.1.2.
  6. Unpack the Atlassian-bomboo-elastic-image-2.2.zip tools into the directory c:\opt\bamboo-elastic-agent\.
  7. Create a user called Bamboo and ensure it has the “Log On as a Service” user-right (see http://technet.microsoft.com/en-us/library/cc739424(v=ws.10).aspx) and it is a member of the groups Users, Remote Desktop Users and IIS_IUSRS (being a member of Remote Desktop Users is a temporary assignment so that the user profile can be initialized).
  8. Edit the file called C:\opt\bamboo-elastic-agent\bin\bamboo-elastic-agent.bat and a line ‘CALL C:\Program Files (x86)\NuGet\StoreNuGetCredentials.cmd’ near the top of batch file so that credentials for private NuGet feeds are initialized into the Bamboo user profile before the bamboo elastic agent is started.
  9. Log off and reconnect as the Bamboo user account to initialize the user profile.
  10. Add a file NuGet.config to the directory C:\Users\Bamboo\AppData\Roaming\NuGet with the following contents to ensure that NuGet packages can be downloaded on demand during builds running under the Bamboo account.
    <?xml version="1.0" encoding="utf-8"?>
    <configuration>
      <packageRestore>
        <add key="enabled" value="True" />
      </packageRestore>
    </configuration>
    
  11. Log off and reconnect as the Administrator again (you can now remove Bamboo from the Remote Desktop Users group if you wish).
  12. Now ensure the Bamboo user account has full access rights to the c:\opt\bamboo-elastic-agent\ directory.
  13. Now launch the “Windows firewall with advanced security” configuration dialog and add a new inbound firewall rule called Bamboo that allows full inbound access for the TCP port 26224 for any remote address.
  14. Download the Service Manager utility for batch files from http://nssm.cc/ and extract the 64-bit nssm.exe to the directory C:\Program Files\Nssm\. We are using a service manager utility to run the bamboo elastic agent rather than a scheduled task (as described in the Atlassian docs) as it is more stable to the sysprep processing performed by Amazon.
  15. Now use the command prompt to ‘”C:\Program Files\Nssm\nssm.exe” install BambooElasticAgent’ to install a service to run the file C:\opt\bamboo-elastic-agent\bin\bamboo-elastic-agent.bat which you specify in the dialog provided.
  16. Now go to the registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BambooElasticAgent and do the following to customize the service installation.
    1. Rename the DisplayName registry value contents from ‘BambooElasticAgent’ to ‘Bamboo Elastic Agent’.
    2. And a string registry value called Description with the contents ‘Bamboo Elastic Agent Service (Using nssm service manager)’.
    3. Add a multi-string registry value called DependOnService with the contents ‘RPCSS’ (This is to ensure the Bamboo Elastic Agent will start after the RPC service).
    4. Under the Parameters sub key create a sub key called AppExit.
    5. In the AppExit sub key ensure the (Default) registry value has the contents ‘Exit’.
    6. In the AppExit sub key create a string registry value called 0 with the contents ‘Ignore’ (This should ensure that the error level return code 0 is ignored by the Nssm service manager).
  17. Go to Administrative Tools – Services and find the Bamboo Elastic Agent service and change the service to run under the account Bamboo with startup type set to Automatic.
  18. Now start the service and ensure that a small file called bamboo-elastic-agent.out has been created in the Bamboo user profile directory with an exception that states the following error ‘This program must be run within an EC2 instance started from Bamboo’.

At this point we should be ready to prepare an Amazon elastic compute image for Bamboo OnDemand to create build server instances from.

Phase 6 – Configure the Amazon EC2 Configuration Service

To get the Amazon EC2 configuration service to sysprep the build server correctly changes have to be made to the file C:\Program Files\Amazon\Ec2ConfigService\SysPrep2008.xml (the changes are marked by the Modification comments).

<?xml version="1.0" encoding="utf-8"?>
<unattend xmlns="urn:schemas-microsoft-com:unattend">
  <settings pass="generalize">
    <component name="Microsoft-Windows-PnpSysprep" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" language="neutral" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" versionScope="nonSxS">
      <PersistAllDeviceInstalls>true</PersistAllDeviceInstalls>
      <DoNotCleanUpNonPresentDevices>true</DoNotCleanUpNonPresentDevices>
    </component>
  </settings>
  <settings pass="oobeSystem">
    <component name="Microsoft-Windows-International-Core" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" language="neutral" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" versionScope="nonSxS">
      <!--Modification Start: Machine will be configured for en-GB rather than the default en-US-->
      <InputLocale>en-GB</InputLocale>
      <SystemLocale>en-GB</SystemLocale>
      <UILanguage>en-GB</UILanguage>
      <UserLocale>en-GB</UserLocale>
      <!--Modification End-->
    </component>
    <component name="Microsoft-Windows-Shell-Setup" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" language="neutral" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" versionScope="nonSxS">
      <OOBE>
        <HideEULAPage>true</HideEULAPage>
        <HideWirelessSetupInOOBE>true</HideWirelessSetupInOOBE>
        <NetworkLocation>Other</NetworkLocation>
        <ProtectYourPC>3</ProtectYourPC>
      </OOBE>
      <BluetoothTaskbarIconEnabled>false</BluetoothTaskbarIconEnabled>
      <!--Modification Start: Machine will be in UK timezone rather than default UTC-->
      <TimeZone>GMT Standard Time</TimeZone>
      <!--Modification End-->
      <RegisteredOrganization>Amazon.com</RegisteredOrganization>
      <RegisteredOwner>Amazon</RegisteredOwner>
    </component>
  </settings>
  <settings pass="specialize">
    <!--Modification Start: New component entry of LUA settings to ensure User Account Control (UAC) is turned off-->
    <component name="Microsoft-Windows-LUA-Settings" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" language="neutral" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" versionScope="nonSxS">
      <EnableLUA>false</EnableLUA>
    </component>
    <!--Modification End-->
    <component name="Microsoft-Windows-Shell-Setup" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" language="neutral" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" versionScope="nonSxS">
      <ComputerName>*</ComputerName>
      <CopyProfile>true</CopyProfile>
      <RegisteredOrganization>Amazon</RegisteredOrganization>
      <!--Modification Start: Machine will be in UK timezone rather than default UTC-->
      <TimeZone>GMT Standard Time</TimeZone>
      <!--Modification End-->
    </component>
    <component name="Microsoft-Windows-Deployment" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" language="neutral" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" versionScope="nonSxS">
      <RunSynchronous>
        <RunSynchronousCommand wcm:action="add">
          <Order>1</Order>
          <Path>net user administrator /ACTIVE:YES /LOGONPASSWORDCHG:NO /EXPIRES:NEVER /PASSWORDREQ:YES</Path>
        </RunSynchronousCommand>
        <RunSynchronousCommand wcm:action="add">
          <Order>2</Order>
          <Path>"C:\Program Files\Amazon\Ec2ConfigService\ScramblePassword.exe" -u Administrator</Path>
        </RunSynchronousCommand>
        <RunSynchronousCommand wcm:action="add">
          <Order>3</Order>
          <Path>sc config ec2config start= auto</Path>
        </RunSynchronousCommand>
        <RunSynchronousCommand wcm:action="add">
          <Order>4</Order>
          <Path>"C:\Program Files\Amazon\Ec2ConfigService\Scripts\SysprepSpecializePhase.cmd"</Path>
        </RunSynchronousCommand>
      </RunSynchronous>
    </component>
    <component name="Microsoft-Windows-ServerManager-SvrMgrNc" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" language="neutral" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" versionScope="nonSxS">
      <DoNotOpenServerManagerAtLogon>true</DoNotOpenServerManagerAtLogon>
    </component>
  </settings>
  <cpi:offlineImage xmlns:cpi="urn:schemas-microsoft-com:cpi" cpi:source="wim:c:/wimbuild/server2008r2x64/install.wim#Windows Server 2008 R2 SERVERDATACENTER" />
</unattend>

Leave all the other Amazon EC2ConfigService settings at their default value.

Now go to part 3 to see how to create the elastic image and connect Bamboo OnDemand to use it.

Posted in Uncategorized | Leave a Comment »

Using Atlassian Bamboo OnDemand for Visual Studio 2012 Projects – Part 1

Posted by David Stringer on 04/10/2012

Introduction

Over the past few months I and my software development colleagues at Quantiv have been looking at Microsoft’s next version of Visual Studio 2012 and it’s associated .Net framework 4.5. Traditionally we have taken a safe Microsoft .Net line and used as much Microsoft tooling as possible to supposedly simplify our software development management processes. Whilst using Visual Studio Team Foundation Server (TFS) 2010 we noticed difficulty in getting our many projects aligned with the TFS way of doing things. This issue was only exacerbated as we moved to Visual Studio 2012 which demanded upgrade to TFS 2012 causing project management compatibility issues. At the same time as dealing with these software process issues, we have also noticed the rise of new development distribution tools, in particular NuGet. This has allowed us to easily experiment with many non-Microsoft promoted technologies such as NoSQL databases like Raven and Mongo which look more ready for the cloud world than traditional technologies such as SQL Server.

As a consequence of all this we are considering a move away from this approach, where we no longer rely on Microsoft to provide the one stop solution. We are now trying out a range of cloud based software development management tools provided by Atlassian to manage some of the software workflow process that TFS would have performed. Atlassian Jira OnDemand has replaced TFS issue tracking and Atlassian Bamboo OnDemand has replaced TFS Build. GitHub has replaced TFS source control as both Jira and Bamboo easily integrate with a Distributed Version Control System (DVCS) such as Git. The attraction of moving to cloud based software development management tools will be hopefully to reduce to cost of ownership of setting up and maintaining the infrastructure to perform software development. I also instinctively like the way these new tools are more loosely coupled so that each one has its own development life cycle that is quite independent of the other. There is somewhat of a problem with the Microsoft software tooling approach where everything has to be released at once, thereby meaning we have to wait for the next version of TFS to get new features.

Creating Custom Elastic Images for Bamboo OnDemand

Atlassian Bamboo OnDemand uses build servers that are hosted in Amazons Ec2 cloud. This article describes how to configure an elastic instance to act as a build server for Visual Studio 2012 projects using GitHub as a source control repository. The article is split into three main parts:

  • Part 1 – Creating a basic elastic instance within Amazon suitable for configuration as a Bamboo OnDemand build server.
  • Part 2 – Configuring the elastic instance with all required build tools and Bamboo elastic agent components so that it can be imaged.
  • Part 3 – Create the elastic instance image and configuring Bamboo OnDemand to connect up to the image, performing test builds and re-imaging an optimized build server.

The first requirement was to create a custom elastic image in Amazon Web Services (AWS) for use with Bamboo OnDemand. At present Bamboo OnDemand only supports bamboo remote agents running as elastic images on the Amazon elastic compute cloud. Company hosted bamboo remote agents are not supported. Furthermore Bamboo OnDemand only supports elastic images in the US East (N. Virginia) region. As our company is based in the UK we would have preferred support for the EU West (Ireland), but this facility is not yet available from Atlassian.

For general documentation regarding Bamboo OnDemand refer to https://confluence.atlassian.com/display/BAMBOO041/Working+with+Elastic+Bamboo.

For general documentation regarding the creation of a custom elastic Windows image refer to https://confluence.atlassian.com/display/BAMBOO041/Creating+a+custom+elastic+image+-+Windows.

To get direct access to the Amazon EC2 management console use the following link https://console.aws.amazon.com/ec2/home?region=us-east-1.

Phase 1 – Launch a suitable amazon base instance in the US East region.

Follow these instructions to configure a suitable amazon base instance that can be used to create the custom elastic image.

  1. In the Amazon EC2 management console (US East region) create a new security group called Bamboo.
  2. Ensure the Bamboo security group has the TCP/IP ports 3389 (RDP) and 26224 (Bamboo elastic agent) added for inbound access by adding a new inbound rule for each of them and ensuring it is saved into the security group.
  3. In the Amazon EC2 management console (US East region) create a key pair called BambooBuilder and save the corresponding private key file (BambooBuilder.pem) to the your local file system.
  4. In the Amazon EC2 management console (US East region) now start the classic wizard for launching an instance by using the “Launch Instance” button on the EC2 dashboard.
  5. On the “Choose an AMI” wizard page select the Microsoft Windows 2008 R2 64-bit Base image.
  6. On the “Instance Details” wizard page select an instance type. I chose High CPU Medium (c1.medium 1.7 GB) as this matched the most suitable option available in Bamboo OnDemand that will be configured later. Normally you will not need to select a specific EC2 availability zone (us-east-1a,…,us-east-1e). Note that for Bambo0 we are launching into EC2 rather than a Virtual Private Cloud (VPC) which is not yet supported by Bamboo OnDemand. Click the continue button when finished.
  7. On the “Instance Details – Advanced Instance Options” wizard page leave all options at their default values and click the continue button. Note that the shutdown behavior you will have selected on this page will be to Stop the instance by default, when Bamboo OnDemand launches instances automatically the shutdown behavior option is set to Terminate instance so that instances are deleted when they are finished with.
  8. On the “Instance Details – Storage Device Configuration” wizard page edit the root volume instance size to something like 40-45Gb (the default is 30Gb which is too small for all the components we will install on the build machine). To keep amazon volume/snapshot storage costs down it is important to tune this value reasonably accurately. The delete on termination flag should be set to true for the root volume. Click the continue button when finished.
  9. On the “Instance Details – Tags wizard” page add any tags that you need to identify the instance. The name tag could be set to something like QuantivNet45BuildServer for our case. Click the continue button when finished.
  10. On the “Create Key Pair” wizard page select the key pair you created earlier called BambooBuilder (or create the key pair at this point if you haven’t done so already). Click the continue button when finished.
  11. On the “Configure Firewall” wizard page select the security group you created earlier called Bamboo (or create the security group at this point if you haven’t done so already). Note that the security group for instance cannot be changed once created. Click the continue button when finished.
  12. Now review the specifications of the instance you are about launch and click the Launch button if you are happy.
  13. The instance will now launch taking about 5 minutes at least until it is ready to be connected to.

For reference on general documentation regarding the launching of instances refer to http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/launching-an-instance.html.

Phase 2 – Connect to the amazon instance

Follow these instructions to connect to the amazon windows instance you have just launched.

  1. In the Amazon EC2 management console (US East region) go to the list of available instances. You may need to use the refresh button on the upper right of the list of instances to ensure your recently launched instance is listed. You can give the instance a name before continuing for easy recognition.
  2. Right click the instance you just launched and select the Connect context menu item.
  3. In the Console Connect – Remote Desktop Connection dialog box that is presented click the retrieve password link.
  4. You will now be asked to upload your private key file BambooBuilder.pem (that you should have downloaded when you created the BambooBuilder key pair earlier).
  5. Once the private key file is uploaded, a Decrypt Password button should be visible which can be clicked.
  6. The Administrator credentials will then be presented to you within the Console Connect – Remote Desktop Connection dialog box which you can copy and paste.
  7. Click the Download shortcut file to save an RDP connection settings file to the location of your choice.
  8. This RDP connection settings file can then be used to connect to the launched instance using the Administrator account with the decrypted password. You’ll get the usual warning about connecting to untrusted machines when using the remote desktop application which you can safely ignore.

For the standard documentation regarding connecting to windows instances refer to http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/LaunchingAndUsingInstancesWindows.html.

Note that every time you restart the instance in the amazon sense (not the windows sense), the instance will come up with a new public DNS address of the form ec2-###-###-###-###.compute-1.amazonaws.com. The public DNS address will only remain stable if you do not stop the amazon instance. This means the RDP connection settings files you have downloaded are redundant once the instance has been stopped as the public DNS address is stale, even though it may be reused by some other instance in the future.

Phase 3 – Basic windows configuration of the amazon instance

Follow these instructions to get the basic windows configuration in place for the new amazon instance. These instructions are fairly specific to the way we build machines in the company I work for, Quantiv. Remember that you can stop the instance following any step below if you need to park your activity, you can then restart the instance via the “My Instances” screen and reconnect via RDP to continue setup.

  1. My preference is to display the control panel items as a sub menu. In order to achieve this you may have to right click the task bar and select properties so that you can customize the start menu. Sometimes the setting will claim the control panel is being displayed as a menu rather than link even though it doesn’t behave that way (this is a minor Windows 7/2008 bug regarding new instances). If you see this bug try setting the control panel to display as a link and apply the changes and then revert it back to display as a menu and apply the changes again and you should fix the problem.
  2. Change windows explorer folder view options to ensure hidden files are shown and file extensions are shown (just my preference).
  3. Go to Control Panel – Region and Language and change all locale settings to English (United Kingdom) including the system locale which will cause a system restart. (Note that a simple windows restart will not change the public DNS address of the amazon instance as the amazon instance has logically remained running).
  4. After reboot reconnect to the amazon instance ensure the language bar is not visible as it just gets in the way.
  5. Go to Control Panel – Date and Time and change the time zone to (UTC) London time.
  6. Go to Control Panel – Programs and Features and choose to turn windows features on or off.
  7. Server Manager should now launch in which you should add the roles Application Server and Web Server. When asked to add the required features generally select everything offered. However the only exception is when configuring the features for the web server do not select FTP server features. Note that we are still installing IIS6 compatibility features just in case we need to use adsi style scripts to control IIS during automatic testing. When asked about SSL certificates for IIS choose the option to import an SSL certificate later.
  8. Change windows update to use Microsoft Update rather than just simple Windows Update. To do this run windows update and click on the link to “Get Updates for other Microsoft products” and agree to the terms and conditions and install Microsoft Updated. Now apply any outstanding updates. Restart if necessary.
  9. Go to Control Panel – User Accounts and select change user account control settings to turn off user account Control (some people may disagree with this change, but it simplifies management of the build box). Wait for yet another restart and reconnect.
  10. Go to Start -> Computer -> Manage -> Configuration -> Local Users and Groups and create an alternative administration account with a known password.
  11. Log out as administrator and reconnect as the alternative administration account to ensure this user profile is primed. Perform steps 1, 2 and 3 for this new account.
  12. You can change the main Administrator password to use the same as the alternative administration account rather than use the automatically generated one to simplify management of building this machine.
  13. Now log out from the alternative administration account account and reconnect as Administrator to continue the installation steps.

In order to make it easier to connect RDP sessions to these dynamic instances I tend to save two RDP connection settings files, one for the Administrator account and one for the alternative administration account. These RDP connection settings files have the computer name blanked out so that when I use them I can paste the temporary computer name obtained from the Amazon EC2 my instances web page for the selected instance I wish to connect to. The RDP connection settings files also do not enable remote printers as this is not an appropriate setting for a build server in the cloud.

Now that you have a basic elastic instance, go to part 2 to see how to configure the build tools and Bamboo elastic agent components.

Posted in Uncategorized | Leave a Comment »