[koalo]

This supersedes #482 by removing the manual mknod procedures with passing udev and /dev directly into the container for the rootful case.

Also reordered the commits as discussed:

1. ELBE without initvm (rootful)

fc80da4
54c6257
da13574

2. Container for ELBE without initvm (rootful)

d532e23
9843820
d7f66a3
707b6c6

3. Test for 2)

ea2994e
ec47114
b5b9d22

4. ELBE without initvm (rootless)

No additional commits, but requires to manually unshare if not done via container.

5. Test XML that allows to build rootless

0095a8d

6. Container for ELBE without initvm (rootless)

43a09d5
41e12bf
76aa59e
79d2fd0

7. Test for 6)

d4cabf0

[t-8ch]

I focused on step 1) for now.
You could also drop the other steps for now and we do one after the other.

[t-8ch]

In the end we want to have a repository/SBOM for the build environment, even if that wasn't an initvm.
We can probably skip it for now, but will need some solution before making this a stable feature.

[koalo]

I also thought of that, but wasn't sure how/if to approach it properly, because I am not so sure if copying the exact same approach from initvm is actually useful for the container use case.

Sure, SBOM generation for the container environment should be done. But for that I guess we should first look how we want to approach SBOM generation for ELBE in the first place (as you mentioned debsbom before).

For the packages, do you think it is worth to assemble the Debian packages from the container standalone like we do for initvm? Wouldn't it be more convenient to just archive the whole container including all the configuration / scripting / ...

[t-8ch]

Wouldn't it be more convenient to just archive the whole container including all the configuration / scripting /

We'll need a machine-readable description of it, not only the raw artifacts.

[koalo]

We'll need a machine-readable description of it, not only the raw artifacts.

Yes, definitely! I meant the "archived container" just as replacement for the binary artifacts of the initvm CD, not as replacement for the SBOM.

[t-8ch]

While I am aware that I asked for this, it is quite complicated...
Maybe we test for the free space instead? That could be done easily in pure python.

[koalo]

Hmm. If you really want it, I can of course implement it. The change for that would be small (see below). But I am not really sure if just checking for the amount of free space is the correct approach. I see two problems with that

1. It introduces an arbitrary constant. How could we know how much space we really need?
2. Even if there is enough space in the RAM for the full build, it still leads to excessive RAM usage, leaving fewer RAM for the build as well as parallel processes. My initial issue that triggered the idea to move to /var/tmp was actually not that the build was failing, but that my browser that I had open in parallel got unusable slow. We can then of course keep some buffer, but again, how much buffer is sufficient?

--- a/conftest.py
+++ b/conftest.py
@@ -2,10 +2,12 @@
 # SPDX-License-Identifier: GPL-3.0-or-later
 # SPDX-FileCopyrightText: 2024 Linutronix GmbH
 
 import os
-import subprocess
+import shutil
 import tempfile
 from pathlib import Path
 
 import pytest
 
+# Minimum free space required in the temp dir for large build artifacts.
+MIN_TMP_FREE = 20 * 1024**3
 
 # https://stackoverflow.com/a/61193490
 def pytest_addoption(parser):
@@ -22,12 +24,6 @@
     )
 
 
-def _is_ram_backed(path):
-    fstype = subprocess.run(
-        ['stat', '-f', '-c', '%T', path],
-        capture_output=True, text=True, check=False,
-    ).stdout.strip()
-    return fstype in ('tmpfs', 'ramfs')
-
-
 def pytest_configure(config):
     config.addinivalue_line('markers', 'slow: mark test as slow to run')
@@ -46,19 +42,22 @@
         os.environ.setdefault('PYTHONWARNINGS', ' '.join(warnings))
 
-    # Avoid filling up the RAM with large build artifacts: unless the user
-    # chose a location, move away from the default temp dir if it is RAM-backed.
+    # Avoid filling up the temp dir (often RAM-backed) with large build
+    # artifacts: unless the user chose a location, move away from the default
+    # temp dir if it has too little free space.
     if 'TMPDIR' not in os.environ and not config.option.basetemp:
-        if _is_ram_backed(Path(tempfile.gettempdir())):
+        if shutil.disk_usage(tempfile.gettempdir()).free < MIN_TMP_FREE:
             var_tmp = Path('/var/tmp')
-            if not var_tmp.is_dir() or not os.access(var_tmp, os.W_OK | os.X_OK):
+            if (
+                not var_tmp.is_dir()
+                or not os.access(var_tmp, os.W_OK | os.X_OK)
+                or shutil.disk_usage(var_tmp).free < MIN_TMP_FREE
+            ):
                 raise pytest.UsageError(
-                    f'{tempfile.gettempdir()} is RAM-backed and {var_tmp} is not a'
-                    ' writable directory. Large build artifacts would fill up the'
-                    ' RAM. Use --basetemp=<dir> or set TMPDIR to select another'
-                    ' location.',
+                    f'{tempfile.gettempdir()} has less than'
+                    f' {MIN_TMP_FREE // 1024**3} GiB free and {var_tmp} is not a'
+                    ' writable directory with enough free space. Use'
+                    ' --basetemp=<dir> or set TMPDIR to select another location.',
                 )
             os.environ['TMPDIR'] = str(var_tmp)
             # tempfile has cached the old location by now, so reset
             tempfile.tempdir = None

[t-8ch]

The database is still stored in /var/cache/elbe. Can we move it into the cache_dir parameter here?

[koalo]

Refactored the whole handling around the cache dir. This was also needed for native builds (because it otherwise resulted in too long socket paths with the gpg agent).

[t-8ch]

Could you mention in the commit message why we need this?