This document is under active development and discussion!

If you find errors or omissions in this document, please don’t hesitate to submit an issue or open a pull request with a fix.

Introduction

WPИ-XM is an open-source web server solution stack for professional PHP development on the Windows® platform.

This manual documents WPN-XM Server Stack internals:

  • Repositories

  • building, packaging and releasing of Installation Wizards

  • Software Registries and API

  • Updater

  • Webinterface

  • Server Control Panel

  • How to contribute to WPN-XM

  • Coding Rules

  • Documentation Toolchain

1. Overview

2. Adding a new component

This chapter describes, how to add a new component to the server stack. Adding a new component is quite complex, because of the dependencies to other components and the adjustments to be made to several files of the WPN-XM environment.

2.1. Add component to software registry

The first step is to add the new component to the software registry.

'component' =>
  array (
    'name' => 'Component',
    'website' => 'http://component.com/',
    '0.0.1' => 'http://component.com/downloads/Component-0.0.1.zip',
    'latest' =>
    array (
      'version' => '0.0.1',
      'url' => 'http://component.com/downloads/Component-0.0.1.zip',
    ),
  ),
  • It doesn’t matter where you insert the new component. The registry array is automatically sorted, when it is updated.

  • You could add all possible "Version" to "URL" relationships, when adding the array. But, it is easier to set only one relationship and have the missing relations inserted automatically (by the version crawler).

Be advised to use a lower version, then the current latest version of the component. This comes handy for testing the version scraping and updating process.

2.2. Add component to Updater

Adding a new crawler

  • You need to add a version crawler class. The crawler classes are located in the crawler folder.

Example-wise we will take a look at the Version Crawler for Adminer:

namespace WPNXM\Updater\Crawler;

use WPNXM\Updater\VersionCrawler;

/**
 * Adminer - Version Crawler
 */
class Adminer extends VersionCrawler (1)
{
  public $url = 'http://www.adminer.org/#download'; (2)

  public function crawlVersion() (3)
  {
    return $this->filter('a')->each(function ($node) { (4)

      if (preg_match("#(\d+\.\d+(\.\d+)*)#", $node->text(), $matches)) { (5)
        $version = $matches[0];

        if (version_compare($version, $this->registry['adminer']['latest']['version'], '>=') === true) { (6)
          return array(
            'version' => $version,
            'url'     => 'https://www.adminer.org/static/download/' . $version . '/adminer-' . $version . '.php',
          ); (7)
        }
      }
    });
  }
}
1 Each Crawler class extends the VersionCrawler base class.
2 The class property $url defines the URL to scrape.
3 Each Crawler class implements the main method: crawlVersion().
4 We are using the dom-selector to filter the content and get closer to the version string.
5 We are working on the filtered $node→text() and grab the version by using a regexp for a version scheme. The matched value is then assigned to $version.
6 We compare the scraped version number with the latest version of our registry. When the scraped version is newer, than the latest version of our registry, we return an array as response.
7 The response is an array with the two keys version and url and their values.

2.2.1. Summary

You added a new component to the registry and to the updater.

  • The registry is synced to the server. This is done by a cronjob. It may take a while, but expect it to be done within 3 to 6 hours.

  • When the new registry comes alive, calls to http://wpn-xm.org/get.php and http://wpn-xm.org/updatecheck.php are possible.

  • You might check the download link status by calling http://localhost/…/updater/registry-status.php.

2.3. Add component to installation wizards

Now that we have the download links available, we add them to the installation wizards.

  • The installation wizards are build from innosetup scripts.

  • They are located in the main project folder of WPN-XM:

  • The are several InnoSetup Script files.

    • wpn-xm-allinone-installer.iss This is the AllInOne Installation Wizards.

    • wpn-xm-webinstaller.iss This is the base for the Webinstallation Wizards.

    • wpn-xm-webinstaller-debug.iss A Webinstallation wizard with enabled Debug mode. You will get some Message Boxes, when switching through the install pages of the wizard.

  • When inserting a new component, we need to modify all of them. You might edit one installer script and transfer the content via a diff tool, like TortoiseGitMerge, to the other files. Take good care, when transfering from a Webinstallation script to the AllInOne installation script. The script files have a big difference, due to the missing download procedures in the AllInOne script.

You can enable the debug mode by setting the #define DEBUG "false" to true.

2.3.1. Adding component to Full Installation Script

  • [Components] section

    • Add the new component to the [Components] section. The components section is the list with checkboxes at the start of the installation wizard, where you can select the components to install.

    • You might use the following line as a template: Name: component; Description: Component - Component does X; ExtraDiskSpaceRequired: 10000; Types: full

    • Please adjust name, description, size.

  • [Files] section

    • You don’t need to add the download file in the [Files] section. All files of the download folder are added automatically.

    • But you might add an additional configuration file needed by the new component.

    • This is done in two steps: Firstly, by adding the configuration file to the configs folder of the WPN-XM main repository: https://github.com/WPN-XM/WPN-XM/tree/master/configs. The file is then copied to the target folder during installation. Secondly, you add the line to copy the file to the [Files] section.

  • [code] section - "const"

    • Add the download filename as a constant to the const section inside the [code] section.

    • Filename_component = 'component.zip';

  • [code] section - "procedure UnzipFiles()".

    • Add a new section for handling the unzipping of your component

if Pos('component', selectedComponents) > 0 then
  begin
    UpdateCurrentComponentName('Component');
      ExtractTemporaryFile(Filename_component);
      DoUnzip(targetPath + Filename_component, ExpandConstant('{app}\bin\component'));
        UpdateTotalProgressBar();
  end;
  • The correct folder is "/www/componentname". Do not use an abbreviation here. Keep it 1:1.

  • If your component doesn’t need to be unzipped, just some file copying, see the handling of APC or XDEBUG, on how to do it.

  • If your zip file contains a component folder inside, you simply unzip to the parent dir: DoUnzip(targetPath + Filename_component, ExpandConstant('{app}\bin'));

  • Sometimes the component folder names are a bit crappy, e.g. component-x86. You might add an additional rename step to the procedure moveFiles() to make it nicer.

if Pos('memcached', selectedComponents) > 0 then
begin
  // rename the existing directory
  Exec('cmd.exe', '/c "move ' + appPath + '\bin\component-x86 ' + appPath + '\bin\component"',
end;
  • Now, during installation, the component will be extracted or copied to the target folder.

  • Additionally, it’s possible to do a configuration step, like modifying the config files of other components. A good example is the modification of php.ini, when you add a new extension. You might add this to the procedure Configure();

    • Add the component to the array in the file "generate-downloads-csv.php"

  • https://github.com/WPN-XM/updater/generate-downloads-csv.php

  • exec script and copy downloads.csv to WPN-XM main folder

2.4. Add component to WPN-XM environment

  • If you added a tool, you need to register it at the "webinterface".

    • It’s located in the "webinterface" repository: https://github.com/WPN-XM/webinterface

    • Update the array $toolDirectories in the file "/webinterface/php/helper/projects.php". https://github.com/WPN-XM/webinterface/blob/master/php/helper/projects.php#L49 This array is used for dividing "Your Project" folders from "WPN-XM Tool" folders.

  • If you added a server, you need to modify the start/stop executables and SCP files.

  • If you added a PHP extension: add the extensions software/php/configs/php.ini, but comment the entry out.

3. Building Installers

This document describes the steps to take, for building a new version of WPN-XM.

Building a new version means to create the installation wizards of WPN-XM.

3.1. Clone the repository

If you prefer to compile your own WPN-XM installer, or if you want to modify it, you will need to obtain a copy of the sources.

Please fetch the repository located at https://github.com/WPN-XM/WPN-XM/, it will give you access to the current development.

You must issue the following command on your command line, DOS box, or terminal:

git clone https://github.com/WPN-XM/WPN-XM/ WPN-XM

This will clone the repository and place it into a directory named WPN-XM.

We try to ensure that the master branch is always stable, but it might occasionally be broken. In that case, take a look at Travis-CI an pick the latest "green" commit.

3.2. Install build-time dependencies

We try to provide a self-contained build environment. You will find all build tools needed for building the installation wizards in the bin folder.

To build WPN-XM on Windows you need the following dependencies:

3.3. How to build the installers automatically

Just execute build.bat. That’s it. It’s a one-click build.

You might then find the setup executable(s) in the folder "_build".

Find a detailed description on how this works in the documentation of the one-click build process.

3.4. How to build the installers manually

Building the installer manually means loading and compiling the InnoSetup Script with Innosetup.

The InnoSetup script files (.iss) reside in the folder WPN-XM/installers.

  • open one of the InnoSetup Script files using InnoSetup or ISTools

  • remove the semicolon from this line: ;#define COMPILE_FROM_IDE to enable local IDE compiling

  • if you need the debug mode, enable it by setting the #define DEBUG "false" to true.

  • Note: the @STACK_VERSION@ token is automatically replaced with the version number during CI builds. But, when working locally and compiling from an IDE the STACK_VERSION string is hard-coded to LocalSnapshot.

  • finally, use the GUI to compile it

    • the webinstallers will compile immediately

    • the packaged installers require their downloads

If everything works correctly this will create the setup executable in the "_build" directory.

3.5. The One-Click-Build Process

This section describes the one-click-build process for the WPN-XM Installer.

build.bat

A build is triggered by executing build.bat in the root folder. The file calls Phing with the build.xml.

Executing build.bat will show all kinds of build information on the screen. For easier debugging the output is also written to a file named build.log.

This requires that you have git on path. Please add the path to git to your environment paths.

Phing

Phing is used build automation tool. It’s a lot like "Ant", but written in PHP. The official website with documentation is https://www.phing.info/ You’ll find a comprehensive guide on over at https://www.phing.info/docs/guide/trunk/

build.xml

Phing is feeded with a build description file. This file is located in the root folder and is named build.xml. The build description file contains properties and the tasks to perform during a build.

Modular Build File and Tasks for WPN-XM

WPN-XM uses a modular build file approach. That means the main build file build.xml includes several smaller build files, which group specific build tasks.

These files reside in the WPN-XM\build-scripts folder:

  • download.build.xml

  • stripdown.build.xml

  • compile-installers.build.xml

Each build file might call PHP tasks to execute specific functionality. These files reside in the WPN-XM\build-scripts\tasks folder:

  • CheckFiles

  • GenerateAriaDownloadLists

  • Download

  • CopyDownloads

  • UpdateComponentSizes

  • CheckInstallers

    • clean-build-dir

"clean-build-dir" recreates the directory _build in the WPN-XM folder. This directory will contain the executables and the modified innosetup script files, which are used for the build.

  • bump-version-number

Uses the APPVERSION property of the build.xml and inserts the current version number into the InnoScripts, Webinterface and Server-Control-Panel files.

  • print-version-number

  • several "compile-wpnxm-setup" and "compile-wpnxm-setup-*" tasks

These tasks perform the compilation of the innosetup scripts into executables, by using the innosetup compiler.

  • compile-wpnxm-webinstaller-setup

Compiles the Web-Installation-Wizard.

  • compile-wpnxm-webinstaller-debug-setup

Compliles the Web-Installation-Wizard with enabled Debug Mode.

  • compile-wpnxm-allinone-installer-no-download

Builds the WPN-XM setup executable using already downloaded components. Assumes that stripdowns are done and components are in the downloads folder. This is for fast rebuilding of the All-In-One Installation Wizard.

  • compile-wpnxm-allinone-installer

This is the main build tasks for building the All-In-One Installation Wizard. This tasks depends on downloading all components and doing stripdowns on MariaDB and MongoDB. Then the WPN-XM setup executable is build.

  • compile-wpnxm-bigpack-installer

  • compile-wpnxm-lite-installer

Several compile tasks. One for each of the innosetup files.

  • "stripdown-mariadb" and "stripdown-mongodb"

Performs a re-packaging after performing a removal of unnecessary files from the zip archives and a compression of executables.

  • "download-components" & "make.downloads.dir"

Uses the "download-filelist.csv" to fetch all software components to the downloads folder.

3.6. Versioning and File Names

The WPN-XM installers use the following naming convention for executables:

WPNXM-<major>.<minor>.<patch>-<installerType>-Setup-php<phpVersion>-w<phpBitsize>.exe, e.g. "WPNXM-0.8.6-Full-Setup-php56-w64.exe".

InstallerType is one of Full, Standard, Lite, Web.

4. Release Process

This chapter documents the release process for WPN-XM.

It is intended for members of the team, who might be cutting a release of WPN-XM.

Releasing WPN-XM is a complicated task.

This document serves as a checklist that can take you through doing a release step-by-step.

We’ll cover the following products:

  1. the WPN-XM server stack installers

  2. the Webinterface

  3. the Server Control Panel

4.1. Prerequisites for doing any release

In order to do a release, you absolutely must have:

  • Commit access to the registry and main WPN-XM repo

While not strictly required, you life will be made much easier if you:

  • Have github account and have been granted access to the WPN-XM organization by a "release admin"

  • Have read and write access to the WPN-XM Travis-CI account

  • Have read and write access to the WPN-XM Appveyor account

4.2. Prerequisites for specific releases

Before getting started, you will need:

  • the new version number, that you will be releasing

  • as well as an agreed upon "release commit" that will form the basis of the release

A commit is eligible to be a "release commit" when:

  • the commit passed all CI checks

  • and building the "installer binaries" succeeded

4.3. Validating "binaries"

4.3.1. Server Control Panel

The binary of the Server Control Panel is validated by executing wpn-xm.exe --version during the Appveyor build and checking the displayed version number.

You find the version output in the Appveyor log of the build.

The version number should look something like:

WPN-XM Server Stack 0.8.3+2cdb580

where the

2cdb580

is the sha for the commit we built from.

4.4. Validate external services are functional

The release process relies on Travis CI and Appveyor as external build and deployment services. Both need to be up and functional in order to do a release.

  • Check the status of each before starting a release. If either is reporting issues, push the release back a day or until whenever they both are reporting no problems.

  • Travis CI Status

  • Appveyor Status

4.5. A GitHub issue exists for the release

All releases should have a corresponding issue in the WPN-XM repo.

This issue should be used to communicate the progress of the release (updated as checklist steps are completed).

You can also use the issue to notify other team members of problems.

4.6. Release Policy

We try to create a stable release a few times a year.

4.7. Release Steps

This section describes the steps to take for releasing a new version of WPN-XM.

4.7.3. WPN-XM Server Stack Installers

4.7.3.1. Update the GitHub issue

Leave a comment on the GitHub issue for this release to let everyone know that you are starting.

4.7.3.2. 1 Update VERSION
  • build.xml

    • adjust version.* properties

  • website-wpn-xm.org/index.html

    • update the download div with new links to Download, Changelog and ReleaseNotes-v1.2.3

    • Do not worry about the version numbers. They are automatically loaded and always in sync with latest versions from registry.

4.7.3.3. 2 Update CHANGELOG

We maintain all changes in our CHANGELOG.

The changes are kept in the unreleased section.

At the time we do a release, we need to "roll" the CHANGELOG from one version to another.

Open up CHANGELOG.md and you will see a series of changes that are in this commit under a section that says:

## [unreleased] - unreleased

This should be changed to:

## [1.2.3] - 2050-01-01

Note: we do not indicate version number changes of the bundled software components. This is automatically published on the website.

If any of the sections, "Added", "Fixed" or "Changed" is empty, delete it.

4.7.3.4. 3 Commit your CHANGELOG and VERSION updates

git add CHANGELOG.md VERSION git commit -m "Preparing for 1.2.3 release"

4.7.3.5. 3 Tag the Version
  • Be sure that all changes for this version are done, before you tag.

  • Repositories: "WPN-XM", "webinterface"

  • Switch to "master" branch: git checkout master.

  • Use git tag -l to determine the upcoming version.

  • Finally use git tag 1.2.3, done.

  • Use git push --tags to push the tags to GitHub.

Travis will build the installers and upload them to Github releases.

(To build the installer locally click "build.bat" in the main folder and find the fresh build in the folder "_build". You might then upload the new installers installation wizards via FTP to the target folder "wpn-xm/downloads".)

4.7.3.6. Wait on Travis and Appveyor

During the time since you push to the release branch, Travis CI and Appveyor have been busy building release artifacts. It can take up to a couple hours depending on how busy they are, before building starts. After building started, it will take up to 40 minutes to build the installers.

4.7.3.7. Announcements

Now, it’s time to spread the word about the new release on multiple channels!

You might use one of the following templates. Please don’t forget to adjust the version number.

 I'm happy to announce the immediate release of WPN-XM v1.2.3.
 For downloads and changelog, see https://github.com/WPN-XM/WPN-XM/releases/latest
 I'm happy to announce the immediate release of WPN-XM v1.2.3.
 For downloads, see: http://wpn-xm.org/downloads
 WPN-XM v1.2.3 has been released! For details, see: http://wpn-xm.org/

Create Release Notes on Github

Use Release-Notes-v1.2.3.md, when manually creating the file. Keep the filename consistent. Use Release Notes v1.2.3 as page title, when creating via the Github Webinterface. The easiest way is to clone an existing file and adjust the file content.

Inform IRC #wpnxm

Drop a note in the #wpnxm IRC channel on Freenode letting everyone know that the release is out and include a link the release notes so everyone can see the CHANGELOG.

Twitter

Tweet about the Release using one of these message templates:

Mailing list

Write a mail to wpn-xm@googlegroups.com or start a new topic via the Google Groups Webinterface.

Community Forum

Post a message on the Community Forum.

5. Software Registries

This chapter covers the software registries of the WPN-XM server stack.

The WPN-XM project maintains multiple registries. A registry is a meta-data collection for multiple software package. It contains pieces of information about the software vendor, the download source and, of course, version information.

The registries are located in the Registry have their own Git repository at https://github.com/WPN-XM/registry.

When we take a look at the repository, you can find 5 registries in the root folder:

Overview Registries
1. `php-extensions-on-pecl.json`        (1)
2. `wpnxm-download-filenames.php`       (2)
3. `wpnxm-php-software-registry.php`    (3)
4. `wpnxm-registry-metadata.php`        (4)
5. `wpnxm-software-registry.php`        (5)
1 A list of all PHP extensions on PECL.
2 Provides download filenames for each software package.
3 A registry for PHP applications.
4 A collection of meta-data for 5.
5 A registry for all software packages of the stack.

Additionally, all released installer have (versionized) installer registries and all unreleased installers have a next version registry.

5.1. Installation Wizards Registries

You will also find multiple registries in the installer folder. The installer registries reside in versionized subfolders. The installer registries for the next, upcoming, but yet unreleased version reside in the installer\next folder.

5.2. Software Registry

The registry for the stack components is always kept in sync with latest versions. Therefore we are monitoring the websites of each software component for newly released versions. If a new version is detected, it is automatically inserted into our registry. In the following sections we explain how the update process of the registry works.

At first, we will take a look at the registries and their underlying data-structure. After that, we take a look at the update mechanism.

5.2.1. WPN-XM Software Registry (wpnxm-php-software-registry.php)

The WPN-XM software registry (wpnxm-php-software-registry.php) is a plain and simple PHP array. The PHP file uses an return array style to make the inclusion of this file and more important the assignment of the data to a variable easy.

$registry = include __DIR__ . '/registry/wpnxm-software-registry.php';

5.2.2. Array Structure

Let’s take a look at the structure of that array.

Example-wise, we take a look at the data structure for the component Adminer:

  'adminer' => array( (1)
    'name' => 'Adminer', (2)
    'website' => 'http://adminer.org/', (3)
    (4)
    '3.5.0' => 'http://garr.dl.sourceforge.net/project/adminer/Adminer/Adminer%203.5.0/adminer-3.5.0.php',
    ...
    '4.1.0' => 'http://garr.dl.sourceforge.net/project/adminer/Adminer/Adminer%204.1.0/adminer-4.1.0.php',
    'latest' => array( (5)
      'version' => '4.1.0',
      'url' => 'http://garr.dl.sourceforge.net/project/adminer/Adminer/Adminer%204.1.0/adminer-4.1.0.php',
    ),
  ),
1 The array key is the shorthand name for the software component. Its allows to use the char - (minus) as separator. A bitsize indication is appended like this: arangodb-x64.
2 The pretty printed name of the software.
3 The URL to the website.
4 One or more version to download URL relationships (1..n).
5 Finally, there is the array key latest, which contains another array as value. This array has the keys version and url and corresponding values.

5.2.3. Accessing Values

To get the details of a component, we can work with the $registry array in the following way:

  // Get "Name" and "Website" for the component
  $name    = $registry['adminer']['name'];
  $website = $registry['adminer']['website'];

  // Get URL for a specific Version
  $url = $registry['adminer']['3.5.0'];

  // Get "Latest Version URL" and "Latest Version"
  $url     = $registry['adminer']['latest']['url'];
  $version = $registry['adminer']['latest']['version'];

5.4. PHP Software Registry (wpnxm-php-software-registry.php)

This file wpnxm-php-software-registry.php is a PHP array.

5.4.1. Array Structure
 'backdrop' => array(
    'name' => 'Backdrop',
    'website' => 'https://backdropcms.org/',
    'description' => 'The comprehensive CMS for small to medium sized businesses and non-profits.',
    'category' => 'cms',
    'tags' => array(
    ),
    'requirements' => array(
      'php-version' => '5.3.2',
      'php-extensions' => ''
    ),
    'source-url' => 'https://github.com/backdrop/backdrop',
    'license' => 'GNU GPL v2',
    'license-url' => 'https://github.com/backdrop/backdrop/blob/1.x/core/LICENSE.txt',
    'docs-url' => 'https://backdropcms.org/handbook',
    'twitter' => array(
      'url' => 'https://twitter.com/backdropcms',
      'hashtag' => '#backdropcms',
    ),
    'logo' => '',
    'screenshots' => array(
    ),
    'versions' => array(
    ),
    'latest' => array(
      'version' => '',
      'url' => '',
    ),
  ),

5.4.2. PHP Extensions on PECL (php-extensions-on-pecl.json)

This file php-extensions-on-pecl.json is a JSON dataset with all extensions on PECL.

It is used for auto-completion of PHP extensions names in the "Install PHP Extension" dialog of the webinterface.

5.4.3. Download Filenames (wpnxm-download-filenames.php)

This file wpnxm-download-filenames.php is a PHP array.

It defines the relationship between

  • a "component" in the software registry (array key)

  • and it’s "filename" in the downloads directory.

Why are we normalizing filenames?

The unmodified download filenames carry a lot of constantly changing pieces of information on their filenames, e.g. version or bitsize information. While hardcoding the original download filenames in the installers is possible, it implies a lot of maintainance effort for keeping them up to date. That’s why we are normalizing the download filenames by removing all changing parts.

We end up with static, normalized filenames. The filenames are used during downloading of components as the "download target" file name.

6. Software Asset Repositories

7. Updater

This chapter describes the Updater, an update and maintainance tool for the software registries and installation wizard registries.

The Update Tool has it’s own Git repository at https://github.com/WPN-XM/updater.

9e38b69e 4233 11e4 9d29 79845ce324a6

7.1. Features

  • CLI and GUI mode

  • fast URL crawling using Guzzle

  • fast version scraping using symfony/dom-crawler, symfony/css-selector, string matching and regular expressions

  • supports version crawling for a single, multiple or all registry component(s)

  • registry health check

7.3. Actions

7.3.1. Action: "Status"

The registry status script (registry-status.php) tests the http access state (200, 404) of the links in the wpnxm-software-registry.php. The test is performed on each components latest version url ($registry[*]['latest']['version']).

The following image shows the output of the dead-links-test. The table column "Download URL" shows the status of latest versions urls on your localhost wpnxm-software-registry.php. The table column "Forwarding URL" shows the status of the links on the server. These links are accessed by the installation wizard.

The results screen after a Registry Status Scan. Here most of the components are green. In most of these cases a 404 indicates that a new version was released by a software vendor. The next step is to scan for a new version and to update the registry accordingly.

registry status

7.3.2. Action: "Scan"

The registry update script (registry-update.php?action=scan) scans vendor websites for newly released versions and updates the wpnxm-software-registry.php accordingly.

7.3.3. Action: "Update"

During a registry scan the newly released versions are stored in to the folder /scans as fragments of the registry. It’s possible to make manual modifications to the data before they are inserted into the registry. After that you might call "Update" to insert the Scans into the registry registry-update.php?action=update.

8. Website

8.1. Installer Comparison View

The installer comparison view is a table showing the comparison between two installers, which are selected by the user. For each installer, you see the software components shipped and the versions. The "change" column indicates, when a software was added, updated or removed. This allows you to quickly notice changes between installers.

8.2. Version Matrix

The version matrix shows all installation wizards and all software components.

This tool helps to avoid version incompatibilities by defining stable version sets of components, which simply work together. It shows already released installers, the current release and the "next" releases. Each installer has it’s own column, while each row contains a software component from our registry.

A green box indicates, that a component is included, also showing it’s version number.

Editor

The editor enables you to create new installer registries.

It allows you to derive installers form already existing ones and to update version numbers of software components by using dropdown menus.

The blue column indicates the "latest version" from the registry.

The installer registries are saved as easy consumable JSON files. They are used in two ways: - to create the download description file - and to provide details about the components included in a release.

The download description file is later used by the download tool to download the components for the packaged installers.

The installer registry json files are used to "show the components included" for a download, for the "installer comparison view" and the "version matrix."

8.3. Configuration Comparison View

This feature isn’t implemented, yet!

The configuration comparison view will display the difference between two versions of a configuration file. Please don’t hesitate to contribute. See GitHub Issue 577.

9. API

You find the API documentation here: http://wpn-xm.org/api-docu/api-doc.htm

Let’s go through the most important API queries: get and updatecheck:

9.1. Get

This is the header redirection script. The script provides a header response to a software and version request in form of a header redirection to the download URL.

The webinstallation wizards use these hard-coded links for downloading components.

Example

Request (GET) for latest version

 https://wpn-xm.org/get.php?s=nginx

Request (GET) for specific version

 https://wpn-xm.org/get.php?s=nginx&v=1.2.1

Response is a header redirection to the download URL.

9.2. Update Check

You want to know, if you are running an old version of a software component and should update?

The updatecheck.php script provides a JSON response to a update-check request for individual components of the WPN-XM Server Stack.

The script accepts two parameters s and v:

  • Parameter s is the name of a software component.

  • Parameter v is the version of this software component.

Example

Request (GET)

https://wpn-xm.org/updatecheck.php?s=nginx&v=1.2.1

Respone (JSON)

{
   "software" : "nginx",
   "your_version" : "1.2.1",
   "latest_version" : "1.3.7",
   "href" : "http:\/\/www.nginx.org\/download\/nginx-1.3.7.zip",
   "message" : "You are running an old version of Nginx and should update immediately."
}

10. Documentation

This chapter describes our documentation toolchain.

10.1. AsciiDoc

The documentation is written in Asciidoc and built by using the Asciidoctor toolchain.

Asciidoc is a human-readable document format, semantically equivalent to DocBook XML, but using plain-text mark-up conventions.

We create the user-manual and this developer-manual in two formats: HTML and PDF.

The HTML output is generated by by asciidoctor. The PDF output is generated by asciidoctor-pdf.

Diagrams are created on-the-fly by asciidoctor-diagram and using ditaa. plantuml or graphviz as renderers.

10.2. Source Folder Layout

The source folder layout supports the translation of manuals.

10.4. Deployment To Github Pages

The deployment to Github Pages is automated.

10.5. Translation

The Asciidoc files are in locale directories: user-manual/XX/ where XX is language (en, fr, de, it, …​).

To start a translation, you would first make a copy of the adoc file source language into the target language. For instance, when translating the Installation chapter from English to Italian, you would copy the English user-manual/en/chapter01-installation.adoc to user-manual/it/chapter01-installation.adoc and then work on it.

The translations missing in files are indicated by this string:

// TRANSLATION MISSING

You must translate whole files, except links and special keywords for notes, warnings. These words must be kept unchanged, because they are Asciidoc Language Syntax and affect output generation:

[[link_name]]
<<link_name>>
[NOTE]
[TIP]
[IMPORTANT]
[WARNING]
[CAUTION]

When there is a name after [link_name], then you must translate it:

<<link_name,this text must be translated>>

11. History

11.2. jakoch’s DevLog

This chapter contains a bit of history and some brief notes about the start and the evolution of this project. Because i tend to forget, how the WPN-XM project started and before all my memories are lost to John Alzheimer, i prefer keeping some. These notes are mainly for myself. If you find them interesting, you are a lucky person.

11.2.1. Before 2010

Eloy Lafuente (stronk7) from the Moodle Project created Moodle for Windows Builder (m4w_builder, GPL). It was a build tool for a webserver stack based on XAMPPlight, including the PHP application Moodle for the developers of the Moodle project. The goal was to provide a quick starter.

I forked it, updated XAMPPlight and integrated our CMS. The goal was to have an out-of-the-box working developer environment. The project was named Clansuite- Serverpack-Builder. It was a subproject of the Clansuite CMS. I made only two releases, which had the same version numbers as the included XAMPPlight version: 1.7.2 and 1.7.3. The builder was batch file based and required many manual release preparation tasks to ship a new version.

11.2.2. 2010

XAMPP releases got somehow stuck: only full releases were made and the XAMPPLight releases, which i relied upon became seldom. I was in a vendor lock- in, where i couldn’t release a new Serverpack, because the main dependency it was based on wasn’t updated itself.

Then suddenly out of nothing the XAMPP project was acquired for an undisclosed amount of money by Bitnami.

Kai Vogelsang stated that things got more and more complicated in managing and releasing the XAMPP stack and it was the right time to give his project in the hands of a company and step back.

I was wondering about this statement and tried to figure out, if it’s really that complicated to create a build system for a server stack. I knew, that 3 guys behind XAMPP have tried to create a Python based build system for it (https://github.com/ApacheFriends/XAMPP-Builder). They got stuck during development and failed to replace their bash-based build system.

I came up with an idea of creating a stack, which partly would solve one of the most important issue on Windows: the not existing package management. The idea was simple. All components which were packaged by installers would have to live in a software components registry, short: the registry.

Because manual maintenance of such a registry is only possible for a few components, i would need to automate the fetching and insertion of latest versions numbers and their URLS. So i started to develop the "Registry Updater", short: the updater. The updater scans the vendor, which provides new releases, fetches any new version number and constructs the new URL for a download and inserts it into the registry.

The next problem i faced was: component incompatibility. I learned the hard way, that some components don’t play well with other components. I came up with the idea of building a version matrix with dropdown form elements for the selection of components and versions, which are known as being compatible. The installer registry was born.

The installer registry is the description file of all components for a certain installer, their download source, their version and their name. By looking at the installer registry file, it’s possible to figure out, which software is included in a certain installer.

I had to make sure that all components got downloaded and were available for inclusion, before the packaging of installers. I tried make the downloads work with curl and wget, but finally ended up working with one of the most amazing download tools i encountered so far: aria2c. It was the first multi-threaded parallel download tool allowing to specify an external file with a set of download URLs and their target folders.

11.2.3. 2011 - 2014

I can’t remember anything. See Changelogs and git log.

11.2.4. 2015

After 3 months (Dec 2014 - Feb 2015) of intensive hacking, i finally got Travis builds and deployment working. This was not easy at all and the first time i build something (the installers) for Windows on Linux under Wine.

But it was more complicated to get the right dependencies for a cross-platform build of the Server Control Panel, which at that time was a Qt5.3+ application. All related toolchain and build issues had quite huge money bounties on them, but i got no feedback and no contributions.

WPN-XM is now the only project with a complete open-source build toolchain for server stacks on Windows. The first releases shipped by the new continuous integration and deployment toolchain was v0.8.1. All 22 installers were build an Travis in under 50 minutes and automatically deployed to Github Releases. We are talking about a release of roughly 3GB of installer executables. Keeping below the build time limit on Travis-CI was a crazy task. I had to tweak memory, iterations and processing and go through everything over and over, just to get below 50 minutes.

11.2.5. 2016

The project is stuck since the release of v0.8.7, because the new features are hard to implement. On the one hand, we have the new process monitoring, PHP spawn handling, the new configuration interface for several components in the SCP, and on the other, the rewrite to short-URLs and security fixes for the webinterface.

I got some contributions, e.g. slashfixes and one-liners. But not a single major feature or bug issue of the Github tracker was solved and implemented by an outside contributor.

Open-source is a morbid software development curiosity, where people are rubbernecking continuously working developers and enjoying the fruits of free and unpaid labor.
— Jens A. Koch

In case, you are still not convinced about using the WPN-XM Server Stack, then go try a different server stack for Windows.

Here’s a comprehensive list for your convenience:

These projects are no longer maintained or updated

General

13. Overview of issue labels

Here is a full list of labels that we use in the GitHub issue tracker and what they stand for.

Label Description

api

Tickets that require changes of API services.

blocker

The ticket cannot be resolved until some other ticket has been closed. See the ticket’s log for which ticket is blocking this ticket.

bounty

This labels marks tickets with compensation for implementing a feature or reporting and closing a bug.

bug

An issue describing unexpected or malicious behaviour of the software.

build tools

This label indicates a problem with one of our build tools or our build toolchain in general.

component related

There is an issue with a bundled component.

configuration

This label marks a configuration issue.

documentation

If an issue involves creating or refining documentation, this label will be assigned.

duplicate bug

easy pick

This label marks tickets that are easy to get started with. The ticket should be ideal for beginners to dive into the code base.

feature

Feature requests and ideas, about how to make the software better, will have this label.

in progress

This label marks tickets, that are not complete yet, but worked upon.

installation wizard

This label indicates an issue with an installation wizard script.

installer registries (next version)

Ticket related to software components in the installer registries of the upcoming, next version.

ready

This label marks feature tickets, which contain enough information to start the implementation.

registry

This label indicates a problem with related to data entries in the installer registries.

registry out-of-sync

This label indicates that an entry in one of the installer registries is no longer up to date or is no longer updated by a crawler.

security

This label indicates a security issue.

server control panel

The ticket marks issues with the bundled Server Control Panel.

support

Questions that needs answering but do not require code changes or issues that only require a one time action on the server will have this label. See the documentation about our triage process for more information.

updater

This label indicates an issue with the Updater.

verify bug

Before the label bug is attached to an issue, the bug needs to be reproduced.

version crawler

This label indicates a problem with one of our version crawlers.

webinterface

The ticket marks issues with the bundled Webinterfaces.

website

The ticket marks issues with the offical website wpn-xm.org.