Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[QE] Explicitly specify CRC_BINARY location while running e2e tests #4629

Open
2 tasks
rohanKanojia opened this issue Feb 19, 2025 · 2 comments · May be fixed by #4633
Open
2 tasks

[QE] Explicitly specify CRC_BINARY location while running e2e tests #4629

rohanKanojia opened this issue Feb 19, 2025 · 2 comments · May be fixed by #4633
Assignees
@rohanKanojia
Labels
kind/task Workable task

Comments

@rohanKanojia
Copy link
Contributor

rohanKanojia commented Feb 19, 2025 •
edited
Loading

Component

Quality Engineering

Task description

Description

Related to devtools-qe-incubator/deliverest#59

During our investigation with @lilyLuLiu and @albfan regarding this intermittent issue. We realized that the E2E pipeline on macos-amd64 was testing an outdated version of CRC binary that was present in PATH.

Pipeline copies the built CRC binary to a location present in PATH and assumes that specific binary would be picked up during test execution. However, if someone was using the test instance and created binary in ~/go/bin this binary would take precedence over the one created by the pipeline. This was causing the intermittent issue of pipeline failing due to presence of outdated binary.

To avoid these kind of issues, we should explicitly specify the location of CRC binary as test execution arguments:

crc/images/build-e2e/lib/darwin/run.sh

Lines 53 to 54 in 457b653

./e2e.test --bundle-location=$bundleLocation --pull-secret-file="${HOME}/$targetFolder/pull-secret" --cleanup-home=false --crc-memory=$crcMemory --godog.tags="$tags" --godog.format=junit > "${HOME}/$targetFolder/results/e2e.results"

Expected Behavior

E2E tests should run with a pinned version of CRC_BINARY

Acceptance Criteria

  • CRC_BINARY flag is added to e2e test execution arguments
  • Pipeline is tested and it's ensured that correct binary is actually being picked up
@rohanKanojia rohanKanojia added the kind/task Workable task label Feb 19, 2025
@rohanKanojia rohanKanojia self-assigned this Feb 19, 2025
@rohanKanojia rohanKanojia moved this to Work In Progress in Project planning: crc Feb 19, 2025
@rohanKanojia rohanKanojia moved this to In Progress in Eclipse JKube Feb 19, 2025
rohanKanojia added a commit to rohankanojia-forks/crc that referenced this issue Feb 19, 2025
@rohanKanojia
test (e2e) : Use pinned version of crc while executing e2e tests (crc…
cdd0fd8 
…-org#4629)

Instead of assuming e2e tests would pick up crc binary added by the pipeline,
that may or may not work if some user has added a binary in any of the PATH
dirs, explicitly specify crc installation dir during test run

Signed-off-by: Rohan Kumar <[email protected]>
@albfan
Copy link
Contributor

albfan commented Feb 19, 2025

While this is desired, explicitly use the PATH for crc binary, we might config a parameter to set it.

If not set, test suite add $HOME/go/bin on top of env PATH

that way we do not harcode any path and allow customizations

rohanKanojia added a commit to rohankanojia-forks/crc that referenced this issue Feb 19, 2025
@rohanKanojia
test (e2e) : Use pinned version of crc while executing e2e tests (crc…
b6462c1 
…-org#4629)

Instead of assuming e2e tests would pick up crc binary added by the pipeline,
that may or may not work if some user has added a binary in any of the PATH
dirs, explicitly specify crc installation dir during test run

Signed-off-by: Rohan Kumar <[email protected]>
@rohanKanojia rohanKanojia linked a pull request Feb 19, 2025 that will close this issue
Draft
12 tasks
@lilyLuLiu
Copy link
Contributor

I suggest to use /usr/local/bin as the default path. As that's where crc is installed during test.

rohanKanojia added a commit to rohankanojia-forks/crc that referenced this issue Feb 20, 2025
@rohanKanojia
test (e2e) : Use pinned version of crc while executing e2e tests (crc…
617b956 
…-org#4629)

Instead of assuming e2e tests would pick up crc binary added by the pipeline,
that may or may not work if some user has added a binary in any of the PATH
dirs, explicitly specify crc installation dir during test run

Signed-off-by: Rohan Kumar <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rohanKanojia rohanKanojia

Labels
kind/task Workable task
Projects
Project planning: crc
Status: Work In Progress
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

test (e2e) : Use pinned version of crc while executing e2e tests (#4629) rohankanojia-forks/crc
3 participants
@albfan @rohanKanojia @lilyLuLiu