summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorScott Rifenbark <scott.m.rifenbark@intel.com>2011-12-07 15:11:32 -0800
committerRichard Purdie <richard.purdie@linuxfoundation.org>2012-01-30 16:05:47 +0000
commite7cfb3b46935ccb8b2c3520fc478aeff61f22779 (patch)
treedb993382b946ab74bc5255dc3c9803723277309a
parentc2494d30148a04ed625e422c4fbc416031cfb178 (diff)
downloadpoky-e7cfb3b46935ccb8b2c3520fc478aeff61f22779.tar.gz
documentation/poky-ref-manual/usingpoky.xml: Removed comments
Removed some comments that were buried in the file that were notes for working on the sstate section. (From yocto-docs rev: bd03315031bbb1b682dcd2253f85fc184822a28e) Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
-rw-r--r--documentation/poky-ref-manual/usingpoky.xml99
1 files changed, 0 insertions, 99 deletions
diff --git a/documentation/poky-ref-manual/usingpoky.xml b/documentation/poky-ref-manual/usingpoky.xml
index eb1f9cb045..1e21c93778 100644
--- a/documentation/poky-ref-manual/usingpoky.xml
+++ b/documentation/poky-ref-manual/usingpoky.xml
@@ -298,107 +298,8 @@
298 <ulink url='http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>. 298 <ulink url='http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>.
299 </note> 299 </note>
300 </section> 300 </section>
301
302
303<!--
304
305 <section id="considering-shared-state-cache">
306 <title>Considering Shared State Cache</title>
307
308 <para>
309 What is shared state in general.
310 Benefits?
311 How we handle things
312 (reference https://lists.yoctoproject.org/pipermail/yocto/2011-March/001157.htm),
313 which is RP's dissertation on how YP solved it.
314 We need to talk a bit about checksum generation for tasks and how the
315 sstate code uses them to figure out what needs rebuilt and what can be re-loaded
316 from the sstate cache.
317 Need to tell about cases where an implicit change can mess things up and under
318 normal situations the state in the sstate cache would be used but it shouldn't be.
319 This is the scenario described by bug 1500 - typical case.
320 Then we talk about how we can invalidate parts of the cache on a per-class basis
321 if needed.
322
323 there is a discussion at
324 https://lists.yoctoproject.org/pipermail/yocto/2011-March/001157.htm
325 that talks about sstate and how the YP team attacked and solved the problem.
326 This is probably a good place to get information from to broach the whole
327 sstate concept.
328
329 YP, by default, builds from scratch.
330 This is good but it means spending a lot of time rebuilding things that don't
331 necessarily need rebuilding.
332
333 The SSTATE_DIR variable points to the directory for the shared state cache that
334 is used during a build.
335
336 A task's inputs have a checksum or signature associated with them.
337 If the checksum changes on an input as compared to a prior build, the task must be rerun.
338 The shared state (sstate) code keeps track of what output is generated by which tasks.
339 So if a task's inputs have not changed then the output associated with the task can
340 be yanked from some place and re-used. No re-build required for that particular task.
341
342 A "run" shell script is created for each task.
343 You can create a checksum for the task based on the inputs to the task.
344 When you have this checksum, the code will look at it and compare it to the previous
345 checksum to see if the task's inputs have changed.
346 If so, the task needs to be re-run.
347
348 Python tasks have python functions that access variables.
349 Python functions will call other python functions as well.
350 The solution was to figure out the variable and function dependencies and create
351 a checksum value for the data coming into the python task.
352
353 Here is a conversation with Mark Hatle regarding bug 1500 (638 is related):
354
355(01:23:34 PM) scottrif: mark - you have a minute?
356(01:34:05 PM) Mark Hatle: sure..
357(01:34:11 PM) Mark Hatle: might be a bit slow to respond, but I'm here
358(01:34:45 PM) scottrif: Hi - I am looking at bug 1500 and trying to get a bit of better understanding. Here is the link to the bug - http://bugzilla.pokylinux.org/show_bug.cgi?id=1500
359(01:35:25 PM) scottrif: It seems that the key for the user here is to when to "Know" when to put some comments into a function to invalidate certain areas of sstate.
360(01:35:49 PM) Mark Hatle: what the issue is, if you make changes to something that is not normally calculated in the checksums for sstate, then you can get package mismatches..
361(01:35:50 PM) scottrif: This trick of "knowing"... does it need to be explained?
362(01:36:25 PM) Mark Hatle: The developer will have to know they made such a change.. Then to deal with this, they can use a patch like what is referenced to cause specific steps to be invalidated and various steps re-run..
363(01:37:01 PM) scottrif: so my question is will the developer know when they make a change like this?
364(01:37:04 PM) Mark Hatle: In this case, we change part of the back-end packaging mechanisms.. which changed internal dependency generation. The sstate code does not checksum the internal dependency generation, it assumes that is code that doesn't change behavior
365(01:37:24 PM) Mark Hatle: They should understand the ramifications of their changes — and thus know they need to do this.
366(01:37:46 PM) Mark Hatle: Examples of times you need to do this. Back end packaging changes occur — i.e. you change the format of dependency generation..
367(01:38:38 PM) scottrif: do you have any other examples?
368(01:38:39 PM) Mark Hatle: when you change a recipe itself, source code.. it is -not- necessary to do this
369(01:38:49 PM) Mark Hatle: RP might be better at examples of when to do it..
370(01:39:11 PM) scottrif: right - If I change a recipe then every thing dependent further down the line gets regenerated right?
371(01:39:17 PM) Mark Hatle: This should never be necessary when a recipe changes.. it will only be necessary when some classes or back-end (packaging frameworks) change..
372(01:39:21 PM) Mark Hatle: ya
373(01:39:33 PM) Mark Hatle: Another way to think of this is implicit dependencies..
374(01:40:01 PM) Mark Hatle: I change RPM.. If you build something that produces an RPM package.. the assumption is the RPM package won't change, even if the RPM binary changes..
375(01:40:10 PM) Mark Hatle: If the format of the package changes.. you would need to do this
376(01:40:53 PM) Mark Hatle: RP can probably give you an idea of the various implicit dependencies, and which ones this type of change is needed for
377(01:41:26 PM) scottrif: okay. I am struggling a bit with how to word it. what I will do is write something up and send it out to you and RP for a look
378(01:41:47 PM) Mark Hatle: ya, I understand.. it's an odd set of situations that can cause this — but we definitely need to document it
379(01:42:01 PM) scottrif: I just want the information to help the user understand the conditions when they will want to invalidate parts of the sstate
380(01:42:18 PM) scottrif: I will likely use the RPM example as the case to illustrate it
381(01:42:26 PM) scottrif: as it seems pretty straight forward
382(01:42:28 PM) Mark Hatle: yup. Key thing is it's only needed on implicit dependencies.. Normal case is back end packaging format changes..
383(01:42:31 PM) Mark Hatle: yup
384(01:42:47 PM) scottrif: ok - thanks Mark
385
386 Here is what RP wants to address 1500:
387
388If its desired to change the checksum of a given subset of tasks, maybe
389due to a change which isn't directly visible in the code itself (e.g. a
390tool changed its output) its possible to do this by changing a function
391comments since the sstate checksums include the body of functions. To
392invalidate package sstate files for example, do_package or one of the
393functions it calls can be changed, even if its just a cosmetic change to
394the commends.
395http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54
396is an example of a commit which does this.
397
398-->
399</section> 301</section>
400 302
401
402<section id='usingpoky-install'> 303<section id='usingpoky-install'>
403 <title>Installing and Using the Result</title> 304 <title>Installing and Using the Result</title>
404 305