1===================================================================
2How To Add Your Build Configuration To LLVM Buildbot Infrastructure
3===================================================================
4
5Introduction
6============
7
8This document contains information about adding a build configuration and
9buildbot-worker to private worker builder to LLVM Buildbot Infrastructure.
10
11Buildmasters
12============
13
14There are two buildmasters running.
15
16* The main buildmaster at `<https://lab.llvm.org/buildbot>`_. All builders
17  attached to this machine will notify commit authors every time they break
18  the build.
19* The staging buildmaster at `<https://lab.llvm.org/staging>`_. All builders
20  attached to this machine will be completely silent by default when the build
21  is broken.
22
23In order to remain connected to the main buildmaster (and thus notify
24developers of failures), a builbot must:
25
26* Be building a supported configuration.  Builders for experimental backends
27  should generally be attached to staging buildmaster.
28* Be able to keep up with new commits to the main branch, or at a minimum
29  recover to tip of tree within a couple of days of falling behind.
30
31Additionally, we encourage all bot owners to point their bots towards the
32staging master during maintenance windows, instability troubleshooting, and
33such.
34
35Roles & Expectations
36====================
37
38Each buildbot has an owner who is the responsible party for addressing problems
39which arise with said buildbot.  We generally expect the bot owner to be
40reasonably responsive.
41
42For some bots, the ownership responsibility is split between a "resource owner"
43who provides the underlying machine resource, and a "configuration owner" who
44maintains the build configuration.  Generally, operational responsibility lies
45with the "config owner".  We do expect "resource owners" - who are generally
46the contact listed in a workers attributes - to proxy requests to the relevant
47"config owner" in a timely manner.
48
49Most issues with a buildbot should be addressed directly with a bot owner
50via email.  Please CC `Galina Kistanova <mailto:[email protected]>`_.
51
52Steps To Add Builder To LLVM Buildbot
53=====================================
54Volunteers can provide their build machines to work as build workers to
55public LLVM Buildbot.
56
57Here are the steps you can follow to do so:
58
59#. Check the existing build configurations to make sure the one you are
60   interested in is not covered yet or gets built on your computer much
61   faster than on the existing one. We prefer faster builds so developers
62   will get feedback sooner after changes get committed.
63
64#. The computer you will be registering with the LLVM buildbot
65   infrastructure should have all dependencies installed and you can
66   actually build your configuration successfully. Please check what degree
67   of parallelism (-j param) would give the fastest build.  You can build
68   multiple configurations on one computer.
69
70#. Install buildbot-worker (currently we are using buildbot version 2.8.5).
71   Depending on the platform, buildbot-worker could be available to download and
72   install with your package manager, or you can download it directly from
73   `<http://trac.buildbot.net>`_ and install it manually.
74
75#. Create a designated user account, your buildbot-worker will be running under,
76   and set appropriate permissions.
77
78#. Choose the buildbot-worker root directory (all builds will be placed under
79   it), buildbot-worker access name and password the build master will be using
80   to authenticate your buildbot-worker.
81
82#. Create a buildbot-worker in context of that buildbot-worker account. Point it
83   to the **lab.llvm.org** port **9990** (see `Buildbot documentation,
84   Creating a worker
85   <http://docs.buildbot.net/current/tutorial/firstrun.html#creating-a-worker>`_
86   for more details) by running the following command:
87
88    .. code-block:: bash
89
90       $ buildbot-worker create-worker <buildbot-worker-root-directory> \
91                    lab.llvm.org:9990 \
92                    <buildbot-worker-access-name> \
93                    <buildbot-worker-access-password>
94
95   To point a worker to silent master please use lab.llvm.org:9994 instead
96   of lab.llvm.org:9990.
97
98#. Fill the buildbot-worker description and admin name/e-mail.  Here is an
99   example of the buildbot-worker description::
100
101       Windows 7 x64
102       Core i7 (2.66GHz), 16GB of RAM
103
104       g++.exe (TDM-1 mingw32) 4.4.0
105       GNU Binutils 2.19.1
106       cmake version 2.8.4
107       Microsoft(R) 32-bit C/C++ Optimizing Compiler Version 16.00.40219.01 for 80x86
108
109#. Make sure you can actually start the buildbot-worker successfully. Then set
110   up your buildbot-worker to start automatically at the start up time.  See the
111   buildbot documentation for help.  You may want to restart your computer
112   to see if it works.
113
114#. Send a patch which adds your build worker and your builder to
115   `zorg <https://github.com/llvm/llvm-zorg>`_. Use the typical LLVM
116   `workflow <https://llvm.org/docs/Contributing.html#how-to-submit-a-patch>`_.
117
118   * workers are added to ``buildbot/osuosl/master/config/workers.py``
119   * builders are added to ``buildbot/osuosl/master/config/builders.py``
120
121   Please make sure your builder name and its builddir are unique through the
122   file.
123
124   All new builders should default to using the "'collapseRequests': False"
125   configuration.  This causes the builder to build each commit individually
126   and not merge build requests.  To maximize quality of feedback to developers,
127   we *strongly prefer* builders to be configured not to collapse requests.
128   This flag should be removed only after all reasonable efforts have been
129   exhausted to improve build times such that the builder can keep up with
130   commit flow.
131
132   It is possible to allow email addresses to unconditionally receive
133   notifications on build failure; for this you'll need to add an
134   ``InformativeMailNotifier`` to ``buildbot/osuosl/master/config/status.py``.
135   This is particularly useful for the staging buildmaster which is silent
136   otherwise.
137
138#. Send the buildbot-worker access name and the access password directly to
139   `Galina Kistanova <mailto:[email protected]>`_, and wait till she
140   will let you know that your changes are applied and buildmaster is
141   reconfigured.
142
143#. Check the status of your buildbot-worker on the `Waterfall Display
144   <http://lab.llvm.org/buildbot/#/waterfall>`_ to make sure it is connected,
145   and the `Workers Display <http://lab.llvm.org/buildbot/#/workers>`_ to see if
146   administrator contact and worker information are correct.
147
148#. Wait for the first build to succeed and enjoy.
149