Chapter 16: Missing layer

[JasonDChistensen]

Chapter 16 page 479 says the bitbake layers should look like the following:

However mine looks like the following:

  core                  /home/jason/poky/meta                                                   5
  yocto                 /home/jason/poky/meta-poky                                              5
  yoctobsp              /home/jason/poky/meta-yocto-bsp                                         5
  openembedded-layer    /home/jason/meta-openembedded/meta-oe                                   5
  meta-python           /home/jason/meta-openembedded/meta-python                               5
  networking-layer      /home/jason/meta-openembedded/meta-networking                           5
  multimedia-layer      /home/jason/meta-openembedded/meta-multimedia                           5
  raspberrypi           /home/jason/meta-raspberrypi                                            9
  meta-mackerel         /home/jason/meta-mackerel                                               6

I know I can remove meta-mackerel, but how do I get the missing layers: filesystems-layer, selinux, webserver ?

Also adding meta-virtualization did not work for me

jason@hp-envy:]$ bitbake-layers layerindex-fetch --branch scarthgap --fetchdir ~meta-virtualization
NOTE: Starting bitbake server...
usage: bitbake-layers layerindex-fetch [-h] [-n] [-b BRANCH] [-s] [-i LAYER] [-f FETCHDIR] layername [layername ...]
bitbake-layers layerindex-fetch: error: the following arguments are required: layername

Here is what chapter 7 (page479) bitbake layer looks like:

[frank-pamir]

@JasonDChistensen there must be a mistake in the bitbake-layers layerindex-fetch command.

Try:

bitbake-layers layerindex-fetch --branch scarthgap --fetchdir ~/meta-virtualization layername meta-virtualization

Or:

bitbake-layers layerindex-fetch --branch scarthgap --fetchdir ~/meta-virtualization layername virtualization

The bitbake-layers layerindex-fetch command will clone the meta-virtualization layer and all its dependent layers including filesystems-layer, selinux, and webserver.

[JasonDChistensen]

The first command did not work for me. Below is the output:

jason@hp-envy:]$ bitbake-layers layerindex-fetch --branch scarthgap --fetchdir ~/meta-virtualization layername meta-virtualization
NOTE: Starting bitbake server...
ERROR: Missing dependency lts-u-boot-mixin (HEAD)
Loading https://layers.openembedded.org/layerindex/api/;branch=scarthgap...
ERROR: Layer "layername" not found in layer index
jason@hp-envy:]$ bitbake-layers show-layers
NOTE: Starting bitbake server...
layer                 path                                                                    priority
========================================================================================================
core                  /home/jason/poky/meta                                                   5
yocto                 /home/jason/poky/meta-poky                                              5
yoctobsp              /home/jason/poky/meta-yocto-bsp                                         5
openembedded-layer    /home/jason/meta-openembedded/meta-oe                                   5
meta-python           /home/jason/meta-openembedded/meta-python                               5
networking-layer      /home/jason/meta-openembedded/meta-networking                           5
multimedia-layer      /home/jason/meta-openembedded/meta-multimedia                           5
raspberrypi           /home/jason/meta-raspberrypi                                            9

The second command did not work for me either. Below is the output:

jason@hp-envy:]$ bitbake-layers layerindex-fetch --branch scarthgap --fetchdir ~/meta-virtualization layername virtualization
NOTE: Starting bitbake server...
ERROR: Missing dependency lts-u-boot-mixin (HEAD)
Loading https://layers.openembedded.org/layerindex/api/;branch=scarthgap...
ERROR: Layer "layername" not found in layer index
ERROR: Layer "virtualization" not found in layer index
jason@hp-envy:]$ bitbake-layers show-layers
NOTE: Starting bitbake server...
layer                 path                                                                    priority
========================================================================================================
core                  /home/jason/poky/meta                                                   5
yocto                 /home/jason/poky/meta-poky                                              5
yoctobsp              /home/jason/poky/meta-yocto-bsp                                         5
openembedded-layer    /home/jason/meta-openembedded/meta-oe                                   5
meta-python           /home/jason/meta-openembedded/meta-python                               5
networking-layer      /home/jason/meta-openembedded/meta-networking                           5
multimedia-layer      /home/jason/meta-openembedded/meta-multimedia                           5
raspberrypi           /home/jason/meta-raspberrypi                                            9

This is the command that worked for me: bitbake-layers layerindex-fetch --branch scarthgap --fetchdir ~/meta-virtualization meta-virtualization

jason@hp-envy:]$ bitbake-layers layerindex-fetch --branch scarthgap --fetchdir ~/meta-virtualization meta-virtualization
NOTE: Starting bitbake server...
ERROR: Missing dependency lts-u-boot-mixin (HEAD)
Loading https://layers.openembedded.org/layerindex/api/;branch=scarthgap...
Layer                                              Git repository (branch)                                 Subdirectory
=============================================================================================================================
local:HEAD:openembedded-core                       git://git.yoctoproject.org/poky.git (scarthgap)         meta
  required by: meta-virtualization meta-filesystems meta-networking meta-python meta-oe meta-selinux meta-webserver
local:HEAD:meta-oe                                 git://git.openembedded.org/meta-openembedded (scarthgap) meta-oe
  required by: meta-python meta-networking meta-filesystems meta-virtualization meta-webserver
local:HEAD:meta-python                             git://git.openembedded.org/meta-openembedded (scarthgap) meta-python
  required by: meta-networking meta-virtualization meta-selinux
local:HEAD:meta-networking                         git://git.openembedded.org/meta-openembedded (scarthgap) meta-networking
  required by: meta-filesystems meta-virtualization
layers.openembedded.org:scarthgap:meta-filesystems git://git.openembedded.org/meta-openembedded (scarthgap) meta-filesystems
  required by: meta-virtualization
layers.openembedded.org:scarthgap:meta-selinux     git://git.yoctoproject.org/meta-selinux (scarthgap)     
  recommended by: meta-virtualization
layers.openembedded.org:scarthgap:meta-webserver   git://git.openembedded.org/meta-openembedded (scarthgap) meta-webserver
  recommended by: meta-virtualization
layers.openembedded.org:scarthgap:meta-virtualization git://git.yoctoproject.org/meta-virtualization (scarthgap) 
Cloning into '/home/jason/meta-virtualization/meta-openembedded'...
remote: Enumerating objects: 313525, done.
remote: Counting objects: 100% (320/320), done.
remote: Compressing objects: 100% (196/196), done.
remote: Total 313525 (delta 156), reused 211 (delta 116), pack-reused 313205 (from 1)
Receiving objects: 100% (313525/313525), 76.63 MiB | 5.52 MiB/s, done.
Resolving deltas: 100% (193792/193792), done.
Updating files: 100% (7055/7055), done.
Cloning into '/home/jason/meta-virtualization/meta-selinux'...
remote: Enumerating objects: 8512, done.
remote: Counting objects: 100% (25/25), done.
remote: Compressing objects: 100% (25/25), done.
remote: Total 8512 (delta 13), reused 0 (delta 0), pack-reused 8487 (from 1)
Receiving objects: 100% (8512/8512), 1.67 MiB | 2.15 MiB/s, done.
Resolving deltas: 100% (5696/5696), done.
Cloning into '/home/jason/meta-virtualization/meta-virtualization'...
remote: Enumerating objects: 22618, done.
remote: Counting objects: 100% (51/51), done.
remote: Compressing objects: 100% (49/49), done.
remote: Total 22618 (delta 19), reused 0 (delta 0), pack-reused 22567 (from 1)
Receiving objects: 100% (22618/22618), 9.08 MiB | 4.10 MiB/s, done.
Resolving deltas: 100% (14377/14377), done.
Adding layer "meta-filesystems" (/home/jason/meta-virtualization/meta-openembedded/meta-filesystems) to conf/bblayers.conf
Adding layer "meta-selinux" (/home/jason/meta-virtualization/meta-selinux/) to conf/bblayers.conf
Adding layer "meta-webserver" (/home/jason/meta-virtualization/meta-openembedded/meta-webserver) to conf/bblayers.conf
Adding layer "meta-virtualization" (/home/jason/meta-virtualization/meta-virtualization/) to conf/bblayers.conf
jason@hp-envy:]$ 
jason@hp-envy:]$ 
jason@hp-envy:]$ bitbake-layers show-layers
NOTE: Starting bitbake server...
layer                 path                                                                    priority
========================================================================================================
core                  /home/jason/poky/meta                                                   5
yocto                 /home/jason/poky/meta-poky                                              5
yoctobsp              /home/jason/poky/meta-yocto-bsp                                         5
openembedded-layer    /home/jason/meta-openembedded/meta-oe                                   5
meta-python           /home/jason/meta-openembedded/meta-python                               5
networking-layer      /home/jason/meta-openembedded/meta-networking                           5
multimedia-layer      /home/jason/meta-openembedded/meta-multimedia                           5
raspberrypi           /home/jason/meta-raspberrypi                                            9
filesystems-layer     /home/jason/meta-virtualization/meta-openembedded/meta-filesystems      5
selinux               /home/jason/meta-virtualization/meta-selinux                            5
webserver             /home/jason/meta-virtualization/meta-openembedded/meta-webserver        5
virtualization-layer  /home/jason/meta-virtualization/meta-virtualization                     8

I commented out the following in conf/local.conf
#DISTRO = "mackerel"

I added the following in conf/local.conf
DISTRO_FEATURES:append = " virtualization"

Then I built the image
bitbake rpi-test-image

For some reason balenaEtcher hung when trying to flash the SD card with rpi-test-image-raspberrypi4-64.rootfs.wic.bz2. It kept getting stuck at uncompressing the file. My work around was to use Ubuntu to extract the file. Then I flashed the file that was extracted.

[JasonDChistensen]

There is a type on page 416:
cd MELD/Chapter15/do-work

It should be:
cd MELD/Chapter14/do-work

---

There is a type on page 516.
conda activate zeromq

It should be:
conda activate pyzmq

[JasonDChistensen]

I could not get the mtrace example to work on page 546. I am running Ubuntu 24.04.4.

Here is the error I was getting:

jason@hp-envy:]$ export MALLOC_TRACE=mtrace.log
jason@hp-envy:]$ ./mtrace-example
jason@hp-envy:]$ mtrace mtrace-example mtrace.log
Cannot open mtrace data file at /usr/bin/mtrace line 152, <locs> line 4.

After some googling I was able to figure out how to make it work.

In recent glibc versions, you must explicitly preload the debug library to enable memory tracing.

jason@hp-envy:]$ LD_PRELOAD=/lib/x86_64-linux-gnu/libc_malloc_debug.so.0 MALLOC_TRACE=./mtrace.log ./mtrace-example
jason@hp-envy:]$ mtrace mtrace-example mtrace.log

Memory not freed:
-----------------
           Address     Size     Caller
0x00005b15239824a0     0x64  at /home/jason/MELD/Chapter18/mtrace-example/mtrace-example.c:11 (discriminator 3)
0x00005b1523982510     0x64  at /home/jason/MELD/Chapter18/mtrace-example/mtrace-example.c:11 (discriminator 3)
0x00005b1523982580    0x100  at /home/jason/MELD/Chapter18/mtrace-example/mtrace-example.c:15

[frank-pamir]

@JasonDChistensen I am busy working on-site in Shenzhen right now through May 21. Please continue to report errata here. I will not respond until I am back in the U.S. May 22.

[JasonDChistensen]

Issue running mtrace in Chapter 18 page 545

Here is the result when I run the mtrace example:

jason@hp-envy:]$ export MALLOC_TRACE=mtrace.log
jason@hp-envy:]$ ./mtrace-example
jason@hp-envy:]$ mtrace mtrace-example mtrace.log
Cannot open mtrace data file at /usr/bin/mtrace line 152, <locs> line 4.

LD_PRELOAD=/usr/lib/libc_malloc_debug.so MALLOC_TRACE=mtrace.log ./mtrace-example

In recent glibc versions, you must explicitly preload the debug library to enable memory tracing. Use the following environment variables when running your compiled program:

jason@hp-envy:]$ LD_PRELOAD=/lib/x86_64-linux-gnu/libc_malloc_debug.so.0 MALLOC_TRACE=./mtrace.log ./mtrace-example
jason@hp-envy:]$ mtrace mtrace-example mtrace.log

Memory not freed:
-----------------
           Address     Size     Caller
0x00005b15239824a0     0x64  at /home/jason/MELD/Chapter18/mtrace-example/mtrace-example.c:11 (discriminator 3)
0x00005b1523982510     0x64  at /home/jason/MELD/Chapter18/mtrace-example/mtrace-example.c:11 (discriminator 3)
0x00005b1523982580    0x100  at /home/jason/MELD/Chapter18/mtrace-example/mtrace-example.c:15