summaryrefslogtreecommitdiffstats
path: root/documentation/getting-started/eclipse/html/getting-started/shared-state.html
diff options
context:
space:
mode:
Diffstat (limited to 'documentation/getting-started/eclipse/html/getting-started/shared-state.html')
-rw-r--r--documentation/getting-started/eclipse/html/getting-started/shared-state.html268
1 files changed, 268 insertions, 0 deletions
diff --git a/documentation/getting-started/eclipse/html/getting-started/shared-state.html b/documentation/getting-started/eclipse/html/getting-started/shared-state.html
new file mode 100644
index 0000000000..4389684f3b
--- /dev/null
+++ b/documentation/getting-started/eclipse/html/getting-started/shared-state.html
@@ -0,0 +1,268 @@
1<html>
2<head>
3<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
4<title>3.3.3. Shared State</title>
5<link rel="stylesheet" type="text/css" href="../book.css">
6<meta name="generator" content="DocBook XSL Stylesheets V1.76.1">
7<link rel="home" href="index.html" title="Getting Started With Yocto Project">
8<link rel="up" href="shared-state-cache.html" title="3.3. Shared State Cache">
9<link rel="prev" href="overview-checksums.html" title="3.3.2. Checksums (Signatures)">
10<link rel="next" href="tips-and-tricks.html" title="3.3.4. Tips and Tricks">
11</head>
12<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="section" title="3.3.3. Shared State">
13<div class="titlepage"><div><div><h3 class="title">
14<a name="shared-state"></a>3.3.3. Shared State</h3></div></div></div>
15<p>
16 Checksums and dependencies, as discussed in the previous
17 section, solve half the problem of supporting a shared state.
18 The other part of the problem is being able to use checksum
19 information during the build and being able to reuse or rebuild
20 specific components.
21 </p>
22<p>
23 The
24 <a class="link" href="../ref-manual/ref-classes-sstate.html" target="_self"><code class="filename">sstate</code></a>
25 class is a relatively generic implementation of how to
26 "capture" a snapshot of a given task.
27 The idea is that the build process does not care about the
28 source of a task's output.
29 Output could be freshly built or it could be downloaded and
30 unpacked from somewhere - the build process does not need to
31 worry about its origin.
32 </p>
33<p>
34 There are two types of output, one is just about creating a
35 directory in
36 <a class="link" href="../ref-manual/var-WORKDIR.html" target="_self"><code class="filename">WORKDIR</code></a>.
37 A good example is the output of either
38 <a class="link" href="../ref-manual/ref-tasks-install.html" target="_self"><code class="filename">do_install</code></a>
39 or
40 <a class="link" href="../ref-manual/ref-tasks-package.html" target="_self"><code class="filename">do_package</code></a>.
41 The other type of output occurs when a set of data is merged
42 into a shared directory tree such as the sysroot.
43 </p>
44<p>
45 The Yocto Project team has tried to keep the details of the
46 implementation hidden in <code class="filename">sstate</code> class.
47 From a user's perspective, adding shared state wrapping to a task
48 is as simple as this
49 <a class="link" href="../ref-manual/ref-tasks-deploy.html" target="_self"><code class="filename">do_deploy</code></a>
50 example taken from the
51 <a class="link" href="../ref-manual/ref-classes-deploy.html" target="_self"><code class="filename">deploy</code></a>
52 class:
53 </p>
54<pre class="literallayout">
55 DEPLOYDIR = "${WORKDIR}/deploy-${PN}"
56 SSTATETASKS += "do_deploy"
57 do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"
58 do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"
59
60 python do_deploy_setscene () {
61 sstate_setscene(d)
62 }
63 addtask do_deploy_setscene
64 do_deploy[dirs] = "${DEPLOYDIR} ${B}"
65 </pre>
66<p>
67 The following list explains the previous example:
68 </p>
69<div class="itemizedlist"><ul class="itemizedlist" type="disc">
70<li class="listitem"><p>
71 Adding "do_deploy" to <code class="filename">SSTATETASKS</code>
72 adds some required sstate-related processing, which is
73 implemented in the
74 <a class="link" href="../ref-manual/ref-classes-sstate.html" target="_self"><code class="filename">sstate</code></a>
75 class, to before and after the
76 <a class="link" href="../ref-manual/ref-tasks-deploy.html" target="_self"><code class="filename">do_deploy</code></a>
77 task.
78 </p></li>
79<li class="listitem"><p>
80 The
81 <code class="filename">do_deploy[sstate-inputdirs] = "${DEPLOYDIR}"</code>
82 declares that <code class="filename">do_deploy</code> places its
83 output in <code class="filename">${DEPLOYDIR}</code> when run
84 normally (i.e. when not using the sstate cache).
85 This output becomes the input to the shared state cache.
86 </p></li>
87<li class="listitem">
88<p>
89 The
90 <code class="filename">do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_IMAGE}"</code>
91 line causes the contents of the shared state cache to be
92 copied to <code class="filename">${DEPLOY_DIR_IMAGE}</code>.
93 </p>
94<div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;">
95<h3 class="title">Note</h3>
96 If <code class="filename">do_deploy</code> is not already in
97 the shared state cache or if its input checksum
98 (signature) has changed from when the output was
99 cached, the task will be run to populate the shared
100 state cache, after which the contents of the shared
101 state cache is copied to
102 <code class="filename">${DEPLOY_DIR_IMAGE}</code>.
103 If <code class="filename">do_deploy</code> is in the shared
104 state cache and its signature indicates that the
105 cached output is still valid (i.e. if no
106 relevant task inputs have changed), then the
107 contents of the shared state cache will be copied
108 directly to
109 <code class="filename">${DEPLOY_DIR_IMAGE}</code> by the
110 <code class="filename">do_deploy_setscene</code> task
111 instead, skipping the
112 <code class="filename">do_deploy</code> task.
113 </div>
114<p>
115 </p>
116</li>
117<li class="listitem">
118<p>
119 The following task definition is glue logic needed to
120 make the previous settings effective:
121 </p>
122<pre class="literallayout">
123 python do_deploy_setscene () {
124 sstate_setscene(d)
125 }
126 addtask do_deploy_setscene
127 </pre>
128<p>
129 <code class="filename">sstate_setscene()</code> takes the flags
130 above as input and accelerates the
131 <code class="filename">do_deploy</code> task through the
132 shared state cache if possible.
133 If the task was accelerated,
134 <code class="filename">sstate_setscene()</code> returns True.
135 Otherwise, it returns False, and the normal
136 <code class="filename">do_deploy</code> task runs.
137 For more information, see the
138 "<a class="link" href="../bitbake-user-manual/setscene.html" target="_self">setscene</a>"
139 section in the BitBake User Manual.
140 </p>
141</li>
142<li class="listitem">
143<p>
144 The <code class="filename">do_deploy[dirs] = "${DEPLOYDIR} ${B}"</code>
145 line creates <code class="filename">${DEPLOYDIR}</code> and
146 <code class="filename">${B}</code> before the
147 <code class="filename">do_deploy</code> task runs, and also sets
148 the current working directory of
149 <code class="filename">do_deploy</code> to
150 <code class="filename">${B}</code>.
151 For more information, see the
152 "<a class="link" href="../bitbake-user-manual/variable-flags.html" target="_self">Variable Flags</a>"
153 section in the BitBake User Manual.
154 </p>
155<div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;">
156<h3 class="title">Note</h3>
157 In cases where
158 <code class="filename">sstate-inputdirs</code> and
159 <code class="filename">sstate-outputdirs</code> would be the
160 same, you can use
161 <code class="filename">sstate-plaindirs</code>.
162 For example, to preserve the
163 <code class="filename">${PKGD}</code> and
164 <code class="filename">${PKGDEST}</code> output from the
165 <a class="link" href="../ref-manual/ref-tasks-package.html" target="_self"><code class="filename">do_package</code></a>
166 task, use the following:
167 <pre class="literallayout">
168 do_package[sstate-plaindirs] = "${PKGD} ${PKGDEST}"
169 </pre>
170</div>
171<p>
172 </p>
173</li>
174<li class="listitem">
175<p>
176 <code class="filename">sstate-inputdirs</code> and
177 <code class="filename">sstate-outputdirs</code> can also be used
178 with multiple directories.
179 For example, the following declares
180 <code class="filename">PKGDESTWORK</code> and
181 <code class="filename">SHLIBWORK</code> as shared state
182 input directories, which populates the shared state
183 cache, and <code class="filename">PKGDATA_DIR</code> and
184 <code class="filename">SHLIBSDIR</code> as the corresponding
185 shared state output directories:
186 </p>
187<pre class="literallayout">
188 do_package[sstate-inputdirs] = "${PKGDESTWORK} ${SHLIBSWORKDIR}"
189 do_package[sstate-outputdirs] = "${PKGDATA_DIR} ${SHLIBSDIR}"
190 </pre>
191<p>
192 </p>
193</li>
194<li class="listitem">
195<p>
196 These methods also include the ability to take a
197 lockfile when manipulating shared state directory
198 structures, for cases where file additions or removals
199 are sensitive:
200 </p>
201<pre class="literallayout">
202 do_package[sstate-lockfile] = "${PACKAGELOCK}"
203 </pre>
204<p>
205 </p>
206</li>
207</ul></div>
208<p>
209 </p>
210<p>
211 Behind the scenes, the shared state code works by looking in
212 <a class="link" href="../ref-manual/var-SSTATE_DIR.html" target="_self"><code class="filename">SSTATE_DIR</code></a>
213 and
214 <a class="link" href="../ref-manual/var-SSTATE_MIRRORS.html" target="_self"><code class="filename">SSTATE_MIRRORS</code></a>
215 for shared state files.
216 Here is an example:
217 </p>
218<pre class="literallayout">
219 SSTATE_MIRRORS ?= "\
220 file://.* http://someserver.tld/share/sstate/PATH;downloadfilename=PATH \n \
221 file://.* file:///some/local/dir/sstate/PATH"
222 </pre>
223<p>
224 </p>
225<div class="note" title="Note" style="margin-left: 0.5in; margin-right: 0.5in;">
226<h3 class="title">Note</h3>
227 The shared state directory
228 (<code class="filename">SSTATE_DIR</code>) is organized into
229 two-character subdirectories, where the subdirectory
230 names are based on the first two characters of the hash.
231 If the shared state directory structure for a mirror has the
232 same structure as <code class="filename">SSTATE_DIR</code>, you must
233 specify "PATH" as part of the URI to enable the build system
234 to map to the appropriate subdirectory.
235 </div>
236<p>
237 </p>
238<p>
239 The shared state package validity can be detected just by
240 looking at the filename since the filename contains the task
241 checksum (or signature) as described earlier in this section.
242 If a valid shared state package is found, the build process
243 downloads it and uses it to accelerate the task.
244 </p>
245<p>
246 The build processes use the <code class="filename">*_setscene</code>
247 tasks for the task acceleration phase.
248 BitBake goes through this phase before the main execution
249 code and tries to accelerate any tasks for which it can find
250 shared state packages.
251 If a shared state package for a task is available, the
252 shared state package is used.
253 This means the task and any tasks on which it is dependent
254 are not executed.
255 </p>
256<p>
257 As a real world example, the aim is when building an IPK-based
258 image, only the
259 <a class="link" href="../ref-manual/ref-tasks-package_write_ipk.html" target="_self"><code class="filename">do_package_write_ipk</code></a>
260 tasks would have their shared state packages fetched and
261 extracted.
262 Since the sysroot is not used, it would never get extracted.
263 This is another reason why a task-based approach is preferred
264 over a recipe-based approach, which would have to install the
265 output from every task.
266 </p>
267</div></body>
268</html>
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2024-05-01 04:56:56 +0000