Development (and Beta Testing) Never Ends

    Follow

    There were (for us) many firsts with the release of Starry Night Pro Plus 7. Exoplanets, data streaming, in-application updates, automatic crash reports - to name just a few. The scope of these new facilities and modernization of the old ones required tons of development time on the app itself. In addition, a lot of development had to be done on our servers too, a topic of a future article.

    There is, however, one other area which has required many hours of development and use of a large number of technologies, most of them necessary to automate the process. This component is now integral to the lifecycle of SN7 and any future versions of our desktop applications. It assures that problems are caught early and that updates are tested before everyone gets them. I am talking about our Starry Night Beta program.

    Mentioning beta testing evokes notions of release candidates, fixed release schedules and long delays between changes in the beta code and their eventual release to the public. We, of course, do our best to adhere to the wisdom of desktop beta testing. However, we also wanted Starry Night Beta to be more agile, and, above all, always alive.

    Among other things, with the new automatic crash reporting and streamlined workflow of dealing with bug reports and user submitted support tickets, we wanted SN7 development to be as responsive to our users as we can make it. We scrutinize every crash report and investigate every bug report. We try and catch problems early. And we want to get relevant improvements and fixes into the software as soon as possible. This is what drove our implementation of in-app updates. And this is what drives our efforts in keeping Starry Night Beta going, even after SN7 has been released - because updates need testing too!

    There are many arms to the beta (sometimes also referred to as "nightly") builds of SN7 and the systems used to get changes as quickly as possible into the hands of our testers. Here they are in their raw, technical jargon. All of the following tasks are fully automated, set in a sequence, and alert the developers of any problems encountered along the way.

    • Mac: checkout from the relevant branch of SN7 git repository
    • Windows: checkout from the relevant branch of SN7 git repository
    • Mac: build SN7
    • Mac: checkout from the relevant branch of Sky Data git repository
    • Mac: pool all the data for Pro Plus into the new Sky Data folder
    • Mac: add in the Manual and User guide to the final application
    • Mac: checkout from the relevant branch of AllSkyImage git repository
    • Mac: add all the AllSkyImage data that needs to be bundled into Pro Plus final app
    • Mac: build the latest Mac telescope plugin and add it to the application
    • Mac: codesign the final application bundle (to satisfy Gatekeeper)
    • Mac: store the freshly built app until relevant build number in our application storage
    • Mac: construct a pkg installer of the application
    • Mac: build a DMG disk image for the pkg installer to make the installation clearer and easier
    • Fire up a very large AWS EC2 instance that is used to quickly compress Windows installation data
    • Mac: rsync final application to the EC2 instance
    • Mac: create a drag-install DMG on Linux using genisoimage and libdmg-hfsplus
    • Mac: push built dmg to S3 for (full) updates
    • Mac: push the new build number to the relevant branch of SN7 git repository
    • Mac: package and push the new symbols file of our application to HockeyApp - the service we use to receive crash reports
    • Mac: read the relevant build release notes, if any, from our SN7 git repository, otherwise use current date stamp as a comment for the beta release
    • Mac: add details about the freshly built application to our release database (size, build number, version number, location of DMG file)
    • Mac: retrieve all active beta builds from the database
    • Mac: go through each previous version and create a delta update between the current and specific previous version (using BinaryDelta from Sparkle project)
    • Mac: for each delta update created, make an entry in our database with information about the update, size, versions involved
    • Mac: push each delta update to S3 for distribution
    • Windows: start a Windows instance on Parallels (we automate Windows building by invoking tasks from Mac to our virtual machine on Parallels)
    • Windows: build and sign SN7
    • Windows: copy Manual and User guide to the app
    • Windows: create a new full build of Windows for local app archive. Use already built Sky Data from our constructed Mac app.
    • Windows: build and add in relevant Windows plugins (telescope, focuser, MaxIm DL)
    • Windows: format release notes so that they can be used in automated xml-powered update building process with wyBuild
    • Windows: prepare specifications for delta update building by writing out an xml file with the latest information
    • Windows: build all delta updates using wyBuild. Let wyBuild upload all the data to our S3 server for distribution.
    • Windows: take the newly created wyUpdate.exe and client.wyc and put them into a folder used to assemble our Windows installer
    • Windows: push the additions to our wyBuild project and new wyUpdate.exe, client.wyc and wyserver.wys files to the relevant branch of SN7 git repository
    • Windows: launch and build our installer by invoking Advanced Installer on our aip project file (this only builds the minimal setup)
    • Windows: package and push the new symbols file of our application to HockeyApp
    • Windows: push the built installer to our EC2 instance
    • Windows: launch 7z compression separately on AllSkyImage and Sky Data on EC2 instance
    • Windows: using winrar on Linux, compress the installer, additional data 7z archives and custom modified SFX file header into Self-Extracting single exe
    • Windows: sign the installer exe with signcode (from github.com/mono project) on Linux to conform with Microsoft Authenticode
    • Windows: copy the built single-exe installer to S3 for distribution
    • Windows: shutdown the Parallels VM
    • Shutdown the EC2 instance
    • Mac: copy the built pkg-based dmg installer to S3 for distribution (this is currently the slowest part of the process which is why it is last)

    Finally, on the distribution chain, we keep things separate between public releases and Starry Night Beta. Applications built for public release are essentially different builds (which is why beta versions and release versions will never again have the same build numbers). Beta builds only get beta updates. The same rule applies for public releases. On Mac, our appcast server reads different entries from our release database (to which information is added during the above build steps). On Windows we keep separate wyserver.wys and wyBuild projects - one for public releases and one for beta.

    And this is just the nightly automated script for a single application (Pro Plus 7)! There is still a lot to be done to improve this system, parallelize some processes and implement failover solutions for others. We are currently using Parallels VM in order to easily script everything directly from one source. However, a Parallels VM could be replaced with another EC2 (Windows) instance. We want to keep the nightly builds no matter what and to make them faster and more robust! Even beta needs testing and development (a beta for Starry Night Beta?).

    Until next time! Clear skies!

    Marko

    Comments

    • Avatar
      Jeffry Turner (DaltonSkyGazer)

      Marko,

       

      Thanks for giving us a peek into the process how the software evolves!  Very interesting, thanks much for your time and efforts, a lot of activity goes on behind the scenes.

       

      Best Regards,

       

      Jeff Turner

    • Avatar
      Marko Kudjerski

      Thanks for the kind words, Jeff!

    • Avatar
      znotne

      amazing

    Powered by Zendesk