diff options
Diffstat (limited to 'documentation')
| -rw-r--r-- | documentation/poky-ref-manual/usingpoky.xml | 99 |
1 files changed, 0 insertions, 99 deletions
diff --git a/documentation/poky-ref-manual/usingpoky.xml b/documentation/poky-ref-manual/usingpoky.xml index 712dcc6fa9..9a338b3c89 100644 --- a/documentation/poky-ref-manual/usingpoky.xml +++ b/documentation/poky-ref-manual/usingpoky.xml | |||
| @@ -298,107 +298,8 @@ | |||
| 298 | <ulink url='http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>. | 298 | <ulink url='http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>. |
| 299 | </note> | 299 | </note> |
| 300 | </section> | 300 | </section> |
| 301 | |||
| 302 | |||
| 303 | <!-- | ||
| 304 | |||
| 305 | <section id="considering-shared-state-cache"> | ||
| 306 | <title>Considering Shared State Cache</title> | ||
| 307 | |||
| 308 | <para> | ||
| 309 | What is shared state in general. | ||
| 310 | Benefits? | ||
| 311 | How we handle things | ||
| 312 | (reference https://lists.yoctoproject.org/pipermail/yocto/2011-March/001157.htm), | ||
| 313 | which is RP's dissertation on how YP solved it. | ||
| 314 | We need to talk a bit about checksum generation for tasks and how the | ||
| 315 | sstate code uses them to figure out what needs rebuilt and what can be re-loaded | ||
| 316 | from the sstate cache. | ||
| 317 | Need to tell about cases where an implicit change can mess things up and under | ||
| 318 | normal situations the state in the sstate cache would be used but it shouldn't be. | ||
| 319 | This is the scenario described by bug 1500 - typical case. | ||
| 320 | Then we talk about how we can invalidate parts of the cache on a per-class basis | ||
| 321 | if needed. | ||
| 322 | |||
| 323 | there is a discussion at | ||
| 324 | https://lists.yoctoproject.org/pipermail/yocto/2011-March/001157.htm | ||
| 325 | that talks about sstate and how the YP team attacked and solved the problem. | ||
| 326 | This is probably a good place to get information from to broach the whole | ||
| 327 | sstate concept. | ||
| 328 | |||
| 329 | YP, by default, builds from scratch. | ||
| 330 | This is good but it means spending a lot of time rebuilding things that don't | ||
| 331 | necessarily need rebuilding. | ||
| 332 | |||
| 333 | The SSTATE_DIR variable points to the directory for the shared state cache that | ||
| 334 | is used during a build. | ||
| 335 | |||
| 336 | A task's inputs have a checksum or signature associated with them. | ||
| 337 | If the checksum changes on an input as compared to a prior build, the task must be rerun. | ||
| 338 | The shared state (sstate) code keeps track of what output is generated by which tasks. | ||
| 339 | So if a task's inputs have not changed then the output associated with the task can | ||
| 340 | be yanked from some place and re-used. No re-build required for that particular task. | ||
| 341 | |||
| 342 | A "run" shell script is created for each task. | ||
| 343 | You can create a checksum for the task based on the inputs to the task. | ||
| 344 | When you have this checksum, the code will look at it and compare it to the previous | ||
| 345 | checksum to see if the task's inputs have changed. | ||
| 346 | If so, the task needs to be re-run. | ||
| 347 | |||
| 348 | Python tasks have python functions that access variables. | ||
| 349 | Python functions will call other python functions as well. | ||
| 350 | The solution was to figure out the variable and function dependencies and create | ||
| 351 | a checksum value for the data coming into the python task. | ||
| 352 | |||
| 353 | Here is a conversation with Mark Hatle regarding bug 1500 (638 is related): | ||
| 354 | |||
| 355 | (01:23:34 PM) scottrif: mark - you have a minute? | ||
| 356 | (01:34:05 PM) Mark Hatle: sure.. | ||
| 357 | (01:34:11 PM) Mark Hatle: might be a bit slow to respond, but I'm here | ||
| 358 | (01:34:45 PM) scottrif: Hi - I am looking at bug 1500 and trying to get a bit of better understanding. Here is the link to the bug - http://bugzilla.pokylinux.org/show_bug.cgi?id=1500 | ||
| 359 | (01:35:25 PM) scottrif: It seems that the key for the user here is to when to "Know" when to put some comments into a function to invalidate certain areas of sstate. | ||
| 360 | (01:35:49 PM) Mark Hatle: what the issue is, if you make changes to something that is not normally calculated in the checksums for sstate, then you can get package mismatches.. | ||
| 361 | (01:35:50 PM) scottrif: This trick of "knowing"... does it need to be explained? | ||
| 362 | (01:36:25 PM) Mark Hatle: The developer will have to know they made such a change.. Then to deal with this, they can use a patch like what is referenced to cause specific steps to be invalidated and various steps re-run.. | ||
| 363 | (01:37:01 PM) scottrif: so my question is will the developer know when they make a change like this? | ||
| 364 | (01:37:04 PM) Mark Hatle: In this case, we change part of the back-end packaging mechanisms.. which changed internal dependency generation. The sstate code does not checksum the internal dependency generation, it assumes that is code that doesn't change behavior | ||
| 365 | (01:37:24 PM) Mark Hatle: They should understand the ramifications of their changes — and thus know they need to do this. | ||
| 366 | (01:37:46 PM) Mark Hatle: Examples of times you need to do this. Back end packaging changes occur — i.e. you change the format of dependency generation.. | ||
| 367 | (01:38:38 PM) scottrif: do you have any other examples? | ||
| 368 | (01:38:39 PM) Mark Hatle: when you change a recipe itself, source code.. it is -not- necessary to do this | ||
| 369 | (01:38:49 PM) Mark Hatle: RP might be better at examples of when to do it.. | ||
| 370 | (01:39:11 PM) scottrif: right - If I change a recipe then every thing dependent further down the line gets regenerated right? | ||
| 371 | (01:39:17 PM) Mark Hatle: This should never be necessary when a recipe changes.. it will only be necessary when some classes or back-end (packaging frameworks) change.. | ||
| 372 | (01:39:21 PM) Mark Hatle: ya | ||
| 373 | (01:39:33 PM) Mark Hatle: Another way to think of this is implicit dependencies.. | ||
| 374 | (01:40:01 PM) Mark Hatle: I change RPM.. If you build something that produces an RPM package.. the assumption is the RPM package won't change, even if the RPM binary changes.. | ||
| 375 | (01:40:10 PM) Mark Hatle: If the format of the package changes.. you would need to do this | ||
| 376 | (01:40:53 PM) Mark Hatle: RP can probably give you an idea of the various implicit dependencies, and which ones this type of change is needed for | ||
| 377 | (01:41:26 PM) scottrif: okay. I am struggling a bit with how to word it. what I will do is write something up and send it out to you and RP for a look | ||
| 378 | (01:41:47 PM) Mark Hatle: ya, I understand.. it's an odd set of situations that can cause this — but we definitely need to document it | ||
| 379 | (01:42:01 PM) scottrif: I just want the information to help the user understand the conditions when they will want to invalidate parts of the sstate | ||
| 380 | (01:42:18 PM) scottrif: I will likely use the RPM example as the case to illustrate it | ||
| 381 | (01:42:26 PM) scottrif: as it seems pretty straight forward | ||
| 382 | (01:42:28 PM) Mark Hatle: yup. Key thing is it's only needed on implicit dependencies.. Normal case is back end packaging format changes.. | ||
| 383 | (01:42:31 PM) Mark Hatle: yup | ||
| 384 | (01:42:47 PM) scottrif: ok - thanks Mark | ||
| 385 | |||
| 386 | Here is what RP wants to address 1500: | ||
| 387 | |||
| 388 | If its desired to change the checksum of a given subset of tasks, maybe | ||
| 389 | due to a change which isn't directly visible in the code itself (e.g. a | ||
| 390 | tool changed its output) its possible to do this by changing a function | ||
| 391 | comments since the sstate checksums include the body of functions. To | ||
| 392 | invalidate package sstate files for example, do_package or one of the | ||
| 393 | functions it calls can be changed, even if its just a cosmetic change to | ||
| 394 | the commends. | ||
| 395 | http://git.yoctoproject.org/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54 | ||
| 396 | is an example of a commit which does this. | ||
| 397 | |||
| 398 | --> | ||
| 399 | </section> | 301 | </section> |
| 400 | 302 | ||
| 401 | |||
| 402 | <section id='usingpoky-install'> | 303 | <section id='usingpoky-install'> |
| 403 | <title>Installing and Using the Result</title> | 304 | <title>Installing and Using the Result</title> |
| 404 | 305 | ||
