This chapter explains the setup and workflow of an maintenance update in the openSUSE way. However, this should not be limited to openSUSE distribution projects but be usable anywhere (the entire workflow or just parts of it).
The goal of the OBS maintenance process is to publish updates for a frozen project, in this example an entire distribution. These updates need to be approved by a maintenance team and the published result must contain documentation about the changes and be applicable in the easiest way by the users. The result is a package repository with additional informations about the solved problems and defined groups of packages to achieve that. Also binary delta data may get generated to reduce the needed download size for the clients.
This figure is giving a basic overview about the project setup and general workflow for a single package and single maintained distribution. It shows the "openSUSE:11.4" project, which is considered to be frozen and not changing at all anymore. The "openSUSE:11.4:Update" projects hosts all officially released updates. It does not build any binary, just gets it sources and binaries from the maintenance incident project via the release process. The incident project is named "openSUSE:Maintenance:IDxxx" in this example, which is under control of the maintenance team. Official updates get built and reviewed here. QA teams are also testing the binaries from here. However, a user can prepare it in the same way in his project and start the maintenance process via doing a "maintenance" request.
This describes all required steps by all involved persons from preparing to releasing a maintenance update.
A user is usually starting to prepare an update by creating a maintenance branch. This is typically done by creating an own maintenance project. Usually multiple released products are affected, so the server can find out which one are maintained by a given source package name, in this example for glibc including checkout via
osc mbranch glibc osc mbranch --checkout glibc
This is equivalent to the api call /source?cmd=branch&package=glibc.
It is also possible to branch only one defined version, if it is known that only one version is affected. In this example the openSUSE:12.1 version:
osc branch --maintenance openSUSE:12.1 glibc osc branch -M -c openSUSE:12.1 glibc
NOTE: both branch commands do support the --noaccess parameter, which will create a hidden project. This may be used when a not yet public known security shall get fixed.
Afterwards the user needs to do the needed modifications. Packages will be built and can be tested. Afterwards he may add informations about the purpose of this maintenance update via
If the source changes contain references to issue trackers (like bugzilla, CVE or FATE) these will be added to the _patchinfo file.
The server will create a full maintenance channel now, in case the user wants to test this as well. After the user has tested, he has to create a maintenancerequest to ask the maintenance team to accept this as official update incident:
On accepting this request all sources of the entire project will get copied to the incident project and be rebuild. The origin project gets usually removed (based on the request cleanup options).
You may submit a package source from a project which is not prepared as maintenance project. That works via the maintenancerequest mechanism by specifing one or more packages from one project. As a consequence it means also that the first testable build will happen in the maintenance incident project. Also the maintenance team need to write the update information on their own.
osc maintenancerequest [ SOURCEPROJECT [ SOURCEPACKAGES RELEASEPROJECT ] ]
The following example is submitting two packages (kdelibs4 and kdebase4) from the project KDE:Devel project as update for openSUSE:12.1
osc maintenancerequest KDE:Devel kdelibs4 kdebase4 openSUSE:12.1
NOTE: it is also possible to specify an existing incident as target via the --incident parameter. The packages will be merged into that existing incident project then.
The maintenance team may start the process (for example because a security issue was reported and the maintenance team decided that a fix is required). In this case the incident gets created via the webui or via the api call
osc createincidient [PROJECT]
osc api /source/PROJECT?cmd=createmaintenanceincident
osc api /source?cmd=createmaintenanceincident&attribute=OBS:Maintenance.
To document the expected work the creation of a patchinfo package is needed. This can be done via
osc patchinfo [PROJECT]
It is important to add bugzilla entries inside of the _patchinfo file. As long these are open bugzilla entries, the bug assignee will see this patchinfoi on his "my work" webui and osc views, so he knows that work is expected for him.
The maintenance incidents are usually managed by a maintenance team. In case the incident got started by a maintainer a maintenance request is targeted towards the defined maintenance project, in our example this is openSUSE:Maintenance. The defined maintainer and reviewers in this project need to decide about this request. In case it gets accepted, the server is creating a subproject with a unique incident ID and copies the sources and build settings to it. The origin project will get removed usually via the cleanup option. This maintenance project is used to build the final packages.
If the maintenance team decides to merge a new maintenance request with an existing incident, they can run the osc rq setincident $REQUESTID $INCIDENT before accepting the request.
The maintenance team may still modify them or the patchinfo data at this point. An outside maintainer can still submit changes via standard submit request mechanism, but direct write permissions are not granted. When the maintenance people are satisfied with the update, they can create a request to release the sources and binaries to the final openSUSE:11.4:Update project.
The release request needs to specify the source and target for each package. In case just the source package or project is specified the api is completing the request on creation time. It is using this based on the source link target of each package and the release information in the repository definitions.
The release process gets usually started via creating a release request. This sets all affected packages to the locked state, which means that all commands for editing the source or triggering rebuilds are not allowed anymore.
The release request typically needs to be approved by QA and other teams as defined in the Update project. In case something gets declined, the necessary changes need to be submitted to the maintenance project and a new release request has to be created.
A unique release ID will be generated and become part of the updateinfo.xml file in the target project on release event. This ID is different from the incident ID and is usually in the style of "YEAR-COUNTER". The counter is strictly increasing on each release. In case of a re-release of the same incident a release counter will be added.
A different naming scheme can be defined via the OBS:MaintenanceIdTemplate attribute value. The release will move all packages to the update project and extend the target package name with the incident ID. Binaries will be moved as well without modification. The exception is the updateinfo.xml which will be modified by replacing its incident id with the release id.
An update should not, but may have an undetected regression. In this case the update needs a re-release. (If another problem shall be fixed a new incident should be created instead.)
If the current update harms the systems, the maintenance team may decide to take it back immediatly. It can be done by removing the patchinfo.ID package container in the Update projects. This will create a new update channel without this update.
To re-open a release incident project, it must get unlocked and marked as open again. Unlocking can be done either via revoking a release request or via explicit unlocking the incident. The explicit unlock via osc:
osc unlock INCIDENT_PROJECT
is also triggering a rebuild to ensure to have higher release numbers and adding the "trigger=maintenance" flags to the release target definitions. Afterwards the project can be edited again and also gets listed as running incident again.
OBS is tracking maintenance work and can be used as a database for future and past updates.
A maintenance incident is started by creating the incident project, either via a developer request or by the maintenance team.
Default content for an incident is added via branch by attribute call:
The release happens on accepting the request by the maintainers of the release project.
OPTIONAL: A maintenance incident can be restarted by
NOTE: The step 1 and 2 may be done via accepting an incident request instead.
The webui shows the running and past incidents when going to the maintenance project (openSUSE:Maintenance in our example). It shows the open requests either for creating or release an incident. Also the open incidents, which are not yet released are visible.
All users need usually just to visit their "my work" screen in webui or osc to see requests or patchinfog were actions of them are expected:
osc my [work]
The following items list some common ways to search for maintenance incidents via the api:
A developer can see the work to be done by him via searching for patchinfos with open bugzilla entries:
/search/package?match=([kind='patchinfo' and issue/[@state='OPEN' and owner/@login='$USER_LOGIN']])
A maintenance coordinator can see requests for doing a maintenance release via searching for open requests with maintenance_incident action against the maintenance project. They are visible in the webui request page of that project or via
/search/request?match=(state/@name='new') and action/@type='maintenance_incident' and action/target/@project='openSUSE:Maintenance')
A maintenance coordinator can see open incidents via searching for incidents project repositories which have a release target with maintenance trigger. Note: this search result is showing all repositories of a matching project.
A maintenance coordinator can see updates which currently are reviewed (for example by a QA team) via
/search/request?match=(state/@name='review') and action/@type='maintenance_release')
A maintenance coordinator can see updates ready to release via searching for open requests with maintenance_release action.
/search/request?match=(state/@name='new') and action/@type='maintenance_release')
An OBS server is using by default a maintenance space defined via the OBS:Maintenance attribute. This must get created on a project where maintenance incident projects should get created below. This project is also defining the default maintenance maintainers and reviewers in it's acl list.
It is possible to have multiple and independent maintenance name spaces, however the maintenance request must be created against this other namespace manually or using a different attribute.
Maintained projects must be frozen, this means no changes in sources or binaries. All updates will be hosted in the defined update project. This project gets defined via the OBS:UpdateProject attribute which must contain a value with the update project name. In addition to this, an attribute to define the active maintenance should also be defined, by default the OBS:Maintained attribute. The osc mbranch command will take packages from this project as a result.
The Update project should be defined as build disabled as well. Also define a project link to the main project and at least one repository building against the main project.