Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test jailer bindmounts to root #5025

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

anthonycorletti
Copy link

@anthonycorletti anthonycorletti commented Feb 4, 2025

Changes

  • gitignore for virtualenv
  • test spawning a microvm when bound at the rootfs, based test loosely on this example

Reason

Closes #1099

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following Developer
Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • I have read and understand CONTRIBUTING.md.
  • I have run tools/devtool checkstyle to verify that the PR passes the
    automated style checks.
  • I have described what is done in these changes, why they are needed, and
    how they are solving the problem in a clear and encompassing way.
  • I have updated any relevant documentation (both in code and in the docs)
    in the PR.
  • I have mentioned all user-facing changes in CHANGELOG.md.
  • If a specific issue led to this PR, this PR closes the issue.
  • When making API changes, I have followed the
    Runbook for Firecracker API changes.
  • I have tested all new and changed functionalities in unit tests and/or
    integration tests.
  • I have linked an issue to every new TODO.

  • This functionality cannot be added in rust-vmm.

@zulinx86 zulinx86 self-requested a review February 5, 2025 14:17
@zulinx86
Copy link
Contributor

Hello @anthonycorletti ,

Thank you for your contribution!

I'll take a further look at this PR later, but could you fix styling issues first? As instructed in the PR checklist, you can run the coding style test using tools/devtool checkstyle. The styling issues detected by it are:

  • UC2 'Signed-off-by' not found in commit message body: sign-off your comments as documented in the CONTRIBUTION.md
  • B6 Body message is missing: include commit messages explaining why the commit is needed, what is done in the commit and how it solves the problem.
  • the following pylint errors
tests/framework/microvm.py:1119:4: C0116: Missing function or method docstring (missing-function-docstring)
tests/framework/microvm.py:1125:4: C0116: Missing function or method docstring (missing-function-docstring)
tests/framework/microvm.py:1127:17: W1514: Using open without explicitly specifying an encoding (unspecified-encoding)
tests/framework/microvm.py:1145:20: W0106: Expression "[self.unmount(mounted_path) for mounted_path in mounts]" is assigned to nothing (expression-not-assigned)

Thanks,

@zulinx86
Copy link
Contributor

You can squash the last two chore commits into a single commit, because the last one ("chore: fix formatting") reverts the change made in the second last ("chore: update gitignore and fix formatting").

Copy link
Contributor

@zulinx86 zulinx86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some formatting changes are mixed in "test: #1099 jailer mount propagation", so could you put those formatting changes into the formatting commit?

@anthonycorletti anthonycorletti force-pushed the main branch 4 times, most recently from 7869e8e to 6940c7f Compare February 11, 2025 21:56
@anthonycorletti
Copy link
Author

thanks @zulinx86! I addressed your suggestions and cleaned up the commits/ formatting. it looks like my checkstyle is passing on my dev machine but not CI

==================================================================================================================================================== test session starts ====================================================================================================================================================
platform linux -- Python 3.12.3, pytest-8.3.4, pluggy-1.5.0 -- /opt/venv/bin/python
cachedir: ../build/pytest_cache
metadata: {'Python': '3.12.3', 'Platform': 'Linux-6.8.0-52-generic-x86_64-with-glibc2.39', 'Packages': {'pytest': '8.3.4', 'pluggy': '1.5.0'}, 'Plugins': {'json-report': '1.5.0', 'metadata': '3.1.1', 'rerunfailures': '14.0', 'timeout': '2.3.1', 'xdist': '3.6.1'}}
EC2 AMI: NA
rootdir: /firecracker/tests
configfile: pytest.ini
plugins: json-report-1.5.0, metadata-3.1.1, rerunfailures-14.0, timeout-2.3.1, xdist-3.6.1
timeout: 300.0s
timeout method: signal
timeout func_only: False
4 workers [14 items]
scheduling tests via WorkStealingScheduling

integration_tests/style/test_repo.py::test_repo_validate_changelog
integration_tests/style/test_gitlint.py::test_gitlint
integration_tests/style/test_markdown.py::test_markdown_style
integration_tests/style/test_python.py::test_python_style[isort]
[gw3] [  7%] PASSED integration_tests/style/test_repo.py::test_repo_validate_changelog
integration_tests/style/test_rust.py::test_rust_order
[gw3] [ 14%] PASSED integration_tests/style/test_rust.py::test_rust_order
integration_tests/style/test_rust.py::test_rust_style
[gw0] [ 21%] PASSED integration_tests/style/test_gitlint.py::test_gitlint
integration_tests/style/test_licenses.py::test_for_valid_licenses
[gw0] [ 28%] PASSED integration_tests/style/test_licenses.py::test_for_valid_licenses
integration_tests/style/test_licenses.py::test_dependency_licenses
[gw2] [ 35%] PASSED integration_tests/style/test_python.py::test_python_style[isort]
integration_tests/style/test_python.py::test_python_pylint
[gw0] [ 42%] PASSED integration_tests/style/test_licenses.py::test_dependency_licenses
integration_tests/style/test_repo.py::test_repo_no_spaces_in_paths
[gw0] [ 50%] PASSED integration_tests/style/test_repo.py::test_repo_no_spaces_in_paths
integration_tests/style/test_python.py::test_python_style[black --config tests/pyproject.toml]
[gw3] [ 57%] PASSED integration_tests/style/test_rust.py::test_rust_style
integration_tests/style/test_swagger.py::test_firecracker_swagger
[gw0] [ 64%] PASSED integration_tests/style/test_python.py::test_python_style[black --config tests/pyproject.toml]
[gw3] [ 71%] PASSED integration_tests/style/test_swagger.py::test_firecracker_swagger
[gw2] [ 78%] PASSED integration_tests/style/test_python.py::test_python_pylint
integration_tests/style/test_repo.py::test_repo_validate_yaml
[gw2] [ 85%] PASSED integration_tests/style/test_repo.py::test_repo_validate_yaml
[gw1] [ 92%] PASSED integration_tests/style/test_markdown.py::test_markdown_style
integration_tests/style/test_markdown.py::test_markdown_internal_links
[gw1] [100%] PASSED integration_tests/style/test_markdown.py::test_markdown_internal_links

-------------------------------------------------------------------------------------------------------------------------------------------------------- JSON report --------------------------------------------------------------------------------------------------------------------------------------------------------
report saved to: ../test_results/test-report.json
=================================================================================================================================================== slowest 10 durations ====================================================================================================================================================
16.93s call     integration_tests/style/test_markdown.py::test_markdown_style
7.00s call     integration_tests/style/test_python.py::test_python_pylint
1.60s call     integration_tests/style/test_rust.py::test_rust_style
1.16s call     integration_tests/style/test_python.py::test_python_style[black --config tests/pyproject.toml]
0.29s call     integration_tests/style/test_licenses.py::test_dependency_licenses
0.17s call     integration_tests/style/test_python.py::test_python_style[isort]
0.14s call     integration_tests/style/test_gitlint.py::test_gitlint
0.10s call     integration_tests/style/test_swagger.py::test_firecracker_swagger
0.05s call     integration_tests/style/test_repo.py::test_repo_validate_yaml
0.02s call     integration_tests/style/test_licenses.py::test_for_valid_licenses
==================================================================================================================================================== 14 passed in 19.89s ====================================================================================================================================================

==================================================================================================================================================== test session starts ====================================================================================================================================================
platform linux -- Python 3.12.3, pytest-8.3.4, pluggy-1.5.0 -- /opt/venv/bin/python
cachedir: ../build/pytest_cache
metadata: {'Python': '3.12.3', 'Platform': 'Linux-6.8.0-52-generic-x86_64-with-glibc2.39', 'Packages': {'pytest': '8.3.4', 'pluggy': '1.5.0'}, 'Plugins': {'json-report': '1.5.0', 'metadata': '3.1.1', 'rerunfailures': '14.0', 'timeout': '2.3.1', 'xdist': '3.6.1'}}
EC2 AMI: NA
rootdir: /firecracker/tests
configfile: pytest.ini
plugins: json-report-1.5.0, metadata-3.1.1, rerunfailures-14.0, timeout-2.3.1, xdist-3.6.1
timeout: 300.0s
timeout method: signal
timeout func_only: False
4 workers [7 items]
scheduling tests via LoadScheduling

framework/properties.py::framework.properties.get_os_version
framework/gitlint_rules.py::framework.gitlint_rules.EndsSigned.validate
framework/properties.py::framework.properties.get_host_os
framework/utils.py::framework.utils.Timeout
[gw3] [ 14%] SKIPPED framework/utils.py::framework.utils.Timeout
[gw2] [ 28%] PASSED framework/properties.py::framework.properties.get_os_version
framework/utils_imdsv2.py::framework.utils_imdsv2.imdsv2_get
[gw1] [ 42%] PASSED framework/properties.py::framework.properties.get_host_os
framework/utils_imdsv2.py::framework.utils_imdsv2.IMDSv2Client.get
[gw2] [ 57%] SKIPPED framework/utils_imdsv2.py::framework.utils_imdsv2.imdsv2_get
[gw1] [ 71%] SKIPPED framework/utils_imdsv2.py::framework.utils_imdsv2.IMDSv2Client.get
[gw0] [ 85%] PASSED framework/gitlint_rules.py::framework.gitlint_rules.EndsSigned.validate
framework/utils_imdsv2.py::framework.utils_imdsv2.IMDSv2Client
[gw0] [100%] SKIPPED framework/utils_imdsv2.py::framework.utils_imdsv2.IMDSv2Client

-------------------------------------------------------------------------------------------------------------------------------------------------------- JSON report --------------------------------------------------------------------------------------------------------------------------------------------------------
report saved to: ../test_results/test-report.json
=================================================================================================================================================== slowest 10 durations ====================================================================================================================================================
0.05s call     framework/gitlint_rules.py::framework.gitlint_rules.EndsSigned.validate
0.00s call     framework/properties.py::framework.properties.get_host_os
0.00s call     framework/properties.py::framework.properties.get_os_version
0.00s setup    framework/properties.py::framework.properties.get_host_os
0.00s setup    framework/utils.py::framework.utils.Timeout
0.00s setup    framework/gitlint_rules.py::framework.gitlint_rules.EndsSigned.validate
0.00s setup    framework/properties.py::framework.properties.get_os_version
0.00s teardown framework/utils.py::framework.utils.Timeout
0.00s teardown framework/utils_imdsv2.py::framework.utils_imdsv2.IMDSv2Client
0.00s call     framework/utils_imdsv2.py::framework.utils_imdsv2.IMDSv2Client.get
=============================================================================================================================================== 3 passed, 4 skipped in 3.22s ================================================================================================================================================
[Firecracker devtool 2025-02-11T23:16:08+01:00] Finished test run ...

Copy link

codecov bot commented Feb 12, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.13%. Comparing base (3ca2fab) to head (6940c7f).
Report is 8 commits behind head on main.

Current head 6940c7f differs from pull request most recent head 39fedf1

Please upload reports for the commit 39fedf1 to get more accurate results.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5025   +/-   ##
=======================================
  Coverage   83.13%   83.13%           
=======================================
  Files         245      245           
  Lines       26651    26651           
=======================================
  Hits        22156    22156           
  Misses       4495     4495           
Flag Coverage Δ
5.10-c5n.metal 83.61% <ø> (ø)
5.10-m5n.metal 83.59% <ø> (+<0.01%) ⬆️
5.10-m6a.metal 82.80% <ø> (ø)
5.10-m6g.metal 79.56% <ø> (ø)
5.10-m6i.metal 83.58% <ø> (-0.01%) ⬇️
5.10-m7g.metal 79.56% <ø> (ø)
6.1-c5n.metal 83.60% <ø> (-0.01%) ⬇️
6.1-m5n.metal 83.58% <ø> (-0.01%) ⬇️
6.1-m6a.metal 82.80% <ø> (ø)
6.1-m6g.metal 79.56% <ø> (+<0.01%) ⬆️
6.1-m6i.metal 83.58% <ø> (ø)
6.1-m7g.metal 79.56% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@zulinx86
Copy link
Contributor

zulinx86 commented Feb 12, 2025

@anthonycorletti

thanks @zulinx86! I addressed your suggestions and cleaned up the commits/ formatting. it looks like my checkstyle is passing on my dev machine but not CI

yeah, not on my dev machine as well.

FAILED integration_tests/style/test_gitlint.py::test_gitlint - AssertionError: Commit message violates gitlint rules: -: UC2 'Signed-off-by' not found in commit message body
  1: T1 Title exceeds max length (205>72): "test: closes #1099 jailer mount propagation by creating mounts for the guest kernel and rootfs and mounting them to the jailer root as mentioned in #1089 Signed-off-by: Anthony Corletti <[email protected]>"
  3: B6 Body message is missing

it looks your commit has a long commit title ( 6940c7f ).

Another nitpicking is not to use a title like "test closes #PR_ID". Keeping the commit title and message self-contained as much as possible will help future readers :)

Copy link
Contributor

@zulinx86 zulinx86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks what you implemented is different from what we want to check, because your test is just checking file existence from outside jail.

If I would reproduce in shell script what you test, it looks like:

# copying kernel and rootfs
$ echo KERNEL >kernel
$ echo ROOTFS >rootfs

# touching kernel and rootfs inside the jail to be used as mount points
$ sudo touch jail/kernel
$ sudo touch jail/rootfs

# checking the above files are created
$ ls jail/kernel
jail/kernel
$ ls jail/rootfs
jail/rootfs

# bind-mounting to the mount points
$ sudo mount --bind kernel jail/kernel
$ sudo mount --bind rootfs jail/rootfs

# checking the file existence from outside jail
$ ls jail/kernel
jail/kernel
$ ls jail/rootfs
jail/rootfs

# start microvm

# checking the file existence from outside jail
$ ls jail/kernel
jail/kernel
$ ls jail/rootfs
jail/rootfs

What we actually want to test is to check that contents of bind-mounted kernel and rootfs are propagated corrrectly inside the jail. So it's like:

# copying kernel and rootfs
$ echo KERNEL >kernel
$ echo ROOTFS >rootfs

# touching kernel and rootfs inside the jail to be used as mount points
$ sudo touch jail/kernel
$ sudo touch jail/rootfs

# bind-mounting to the mount points
$ sudo mount --bind kernel jail/kernel
$ sudo mount --bind rootfs jail/rootfs

# start microvm
# HERE, mock the jailer and this shell is inside the jail.
$ sudo unshare --mount --propagation unchanged
$ mount --make-rslave /
$ mount --rbind jail jail
$ cd jail/
# Just for simplicity, not pivot_root here.

# contents check
$ cat kernel
KERNEL
$ cat rootfs
ROOTFS

If it were the previous jailer that doesn't support bind-mounts, the result would be that the contents are not propagated properly as follows:

$ echo KERNEL >kernel
$ echo ROOTFS >rootfs
$ sudo touch jail/kernel
$ sudo touch jail/rootfs
$ sudo mount --bind kernel jail/kernel
$ sudo mount --bind rootfs jail/rootfs

# start microvm
# HERE mock preivous jailer (not supporting recursive bind-mounts)
$ sudo unshare --mount --propagation unchanged
# "MS_PRIVATE | MS_REC" instead of "MS_SLAVE | MS_REC"
$ mount --make-rprivate /
# "MS_BIND" instead of "MS_BIND | MS_REC"
$ mount --bind jail jail
$ cd jail/
# Just for simplicity, not pivot_root here.

# the contents of the kernel and rootfs are empty
$ cat kernel
$ cat rootfs

One of my dumb proposals is like:

  1. Make the default test_microvm.kernel_file and test_microvm.rootfs_file empty to make the microvm fail to boot if the contents of bind-mounted kernel and rootfs are not propagated correctly.
  2. Bind-mount new kernel and rootfs onto the paths of test_microvm.kernel_file and test_microvm.rootfs_file.
  3. Test the guest inside the microvm boots and responds to remote command execution from host.

This might not work since I don't test it on my side, so please fix as you think best.

It would be super great if you could come up with a better idea!!

Comment on lines 673 to 674
This is a test for
https://github.com/firecracker-microvm/firecracker/pull/#1093
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make the comment self-contained and not refer to a PR.

.gitignore Outdated
@@ -15,3 +15,4 @@ test_results/*
/resources/linux
/resources/x86_64
/resources/aarch64
.venv
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this for running the python test on your own virtual env? If so, may I ask why you can't use tools/devtool test?

@anthonycorletti anthonycorletti force-pushed the main branch 2 times, most recently from e717ef4 to 89c9346 Compare February 14, 2025 18:11
this test creates two mounts for a rootfs and guest kernel in
directories that are mounted in a location that the jailer
requires to boot successfully
Signed-off-by: Anthony Corletti <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add jailer bind mounts integration test
2 participants