Unable to clean modules installed by xcaddy build

This issue has been created since 2022-06-07.

While building with xcaddy with Yocto I am getting the following error

ERROR: Command '['rm', '-rf', '/work/yocto/build/tmp/work/aarch64-poky-linux/caddy/2.4.6-r0/build/pkg']' returned non-zero exit status 1.

In our build machine, we need write access to modules directory in order to re-compile the project but go modules are installed with read access, only. This can be fixed by executing go build with flag '-modcacherw` but xcaddy does not support passing custom flags from environment.

akeskimo wrote this answer on 2022-06-07

The flag modcacherw has been introduced since go1.4 and sets write bit on pulled go modules. Certain build tools will try to clean up the modules directory and do not have root priviledges so it would be convenient to add this support for xcaddy.

mholt wrote this answer on 2022-06-07

Why do you need write access to the modules? 🤔 What is the xcaddy command you are running and full output?

akeskimo wrote this answer on 2022-06-09

The command I am executing is

xcaddy build e7457b43e4703080ae8713ada798ce3e20b83690 --with github.com/ggicci/[email protected]

which works just fine for the first time. It fails in consequent runs because the build system must clean the files before it can be re-compiled but it fails because go installs modules as RO which causes rm to bail out since my user has no root privileges.

akeskimo wrote this answer on 2022-06-09

There is also another related issue with xcaddy is that Yocto trims the produced binaries automatically but xcaddy enforces this by adding flags


cmd.Args = append(cmd.Args,
	"-ldflags", "-w -s", // trim debug symbols

so I will need to add the following exception to my recipe

# Binaries procuded by xcaddy are stripped already which would cause QA error during build so we can safely ignore this warning.
INSANE_SKIP_${PN} += "already-stripped"

This could be avoided if there would be an environment variable to pass the build flags to xcaddy which would allow the user to control the behavior of compilation in contrast to hardcoding 'em.

mholt wrote this answer on 2022-06-09

It fails in consequent runs because the build system must clean the files before it can be re-compiled

Sorry, I don't understand. Go doesn't delete files to re-compile a second time.

This sounds like more of a problem with whatever Yocto is, than xcaddy or Go. 🤔

akeskimo wrote this answer on 2022-06-10

Sorry, I don't understand. Go doesn't delete files to re-compile a second time.

Yocto is a distributed build-system to compile Linux kernels which are used by millions of Embedded Linux devices. Yes, you are correct: Go does not delete files during recompiling, but a build system might.

This sounds like more of a problem with whatever Yocto is, than xcaddy or Go. thinking

No, this is not an issue with Go or Yocto. Let me explain.

Go installs modules with read-only access (by default) because Golang was developed by Google who uses monolithic repository structure which can benefit from using global module path. The global module install cannot be used for distributed build-systems because modules need to declare their own dependencies which would blow up in disk space. Fortunately, this has been addressed by Golang maintainers a while ago by introducing flag modcacherw which xcaddy seems to be ignoring, for some reason.

Here is background information and discussion related to why the flag is useful golang/go#27161.

akeskimo wrote this answer on 2022-06-10

In Caddy-speak, what I need is something like this

env XCADDY_GOBUILDFLAGS="-modcacherw" xcaddy build e7457b43e4703080ae8713ada798ce3e20b83690 --with github.com/ggicci/[email protected]

which allows me to redefine build arguments from our build system.

I have already patched this in our build but I decided to bring this up because the caddy community would benefit from it.

mholt wrote this answer on 2022-06-11

Oh, I see what you mean now. Thanks for the explanation. Yeah, that env var seems like a good way to work around it, I guess. (I'm way less picky about xcaddy than I am about caddy anyway.) Want to submit a PR?

akeskimo wrote this answer on 2022-06-11

Sure, the patch needs a bit of polishing but I should be able to submit it by next week.

akeskimo wrote this answer on 2022-06-16

I am not sure what is your process for contributing so I created a fork for the pull request. You can comment on the PR if any changes are needed.

mholt wrote this answer on 2022-06-16

Perfect, thanks @akeskimo !

akeskimo wrote this answer on 2022-08-13

Hi @mholt,

We still have the problem that go modules that are pulled in by go get or go mod might be read-only which we are unable to clean from our build environment. I have added another pull request which should fully resolve this issue. We currently have this patch applied in our build system (Yocto) but here it is if you would like to have this in upstream, as well.

More Details About Repo
Owner Name caddyserver
Repo Name xcaddy
Full Name caddyserver/xcaddy
Language Go
Created Date 2020-03-21
Updated Date 2022-09-22
Star Count 381
Watcher Count 14
Fork Count 72
Issue Count 3


Issue Title Created Date Comment Count Updated Date
Add artemislena.eu and gtrr.artemislena.eu 1 2021-07-04 2022-07-21
please remove homburg.xyz 1 2021-09-07 2022-08-31
Update for editions-du-26-octobre 2 2021-06-20 2022-07-23
[CRASH] Unable to start Arclight with Sneaky Magic installed 7 2022-03-14 2022-09-25
cannot load flash programming algorithm 3 2020-07-05 2022-08-31
http service的ssl问题 0 2021-07-23 2022-01-21
定时器文档和时间情况不符合 1 2021-06-01 2022-01-23
The mod doesn't seem to work 12 2020-04-22 2022-01-01
Cannot move camera ingame. 0 2021-06-20 2022-01-01
Camera/Player perspective is affected by mouse movement in UI menus 1 2021-07-20 2022-01-01
Support Windows 11 1 2021-11-29 2022-01-19
latest version of cvodes documentation 0 2022-05-24 2022-09-11
Building a threat model 1 2022-04-08 2022-05-14
calibre crash on a large library 3 2021-07-10 2022-09-27
ssl port alternative. 2 2021-07-10 2022-08-30
Mermaid pie chart does not export properly to PDF 0 2021-09-09 2022-09-04
Disabled automatic scanning 4 2021-01-31 2022-09-02
Yoast Video SEO: Investigate compatibility 1 2022-01-26 2022-08-31
Allow inputing dates in forms 0 2021-10-16 2022-08-12
vcn LC use cases: check LC version as first step 2 2020-11-04 2022-08-18
Bug: Application error: Uncaught (in promise): CloseEvent: {"isTrusted":true} 6 2022-05-04 2022-09-18
External Link page save error, when coming from a different page 0 2022-06-27 2022-08-27
No audio after suspension until program is closed 11 2021-08-10 2022-09-10
Error when attempting to reset zoom 1 2021-06-23 2022-09-25
cuda::videoWriter and VideoWriter error 10 2021-12-06 2022-09-23
core: Is GlobalLoggingInitCall necessary? 5 2021-12-13 2022-09-19
dist-spec 1.0.0-rc OPTIONAL marked items 16 2020-10-09 2022-09-28
Exception in thread "main" java.lang.UnsupportedClassVersionError: org/contikios/cooja/Cooja : Unsupported major.minor version 52.0 2 2017-09-20 2022-09-25
Array of objects - JSON body as request payload 12 2021-09-05 2022-08-23
Extending RTAB-Map with additional sensor data 2 2021-08-23 2022-09-15
get_script_method_list() output differs from get_method_list() on same script and neither return 'default_args[]' 2 2021-11-30 2022-08-28
Feature request: Implement Azure AD support 1 2021-05-11 2022-07-22
Java imports cleanup 0 2021-11-15 2022-09-16
Course preview in dashboard 1 2018-11-30 2022-09-15
Heap memory increasing issue.. 4 2021-11-13 2022-09-08
Remove / re-work latex extension examples when mathext labels land 1 2021-09-14 2022-08-21
OOM with JAX-RS streaming output 6 2022-07-29 2022-09-13
Re-use generate params file code between "bicep deploy" and "generate parameters" features 10 2022-06-07 2022-09-17
LUIS and QnA maker dispatcher is slow 1 2021-07-03 2022-09-06
Taking over .... 15 2013-10-12 2022-01-12
DictionaryPath ignores XDG_CONFIG_HOME 0 2021-09-30 2022-08-08
AWSSDK.Extensions.NETCore.Setup doesn't work correctly 6 2021-09-03 2022-09-25
[DOC] Please resurrect the targeted 'Integrate django CMS into an existing project' page. 3 2021-11-22 2022-09-25
Fix race conditions in EntityActionScheduler 0 2021-07-02 2022-09-08
Perhaps use Vert.x message headers to hold AMQP properties 0 2015-06-04 2022-07-18
[Bug]: ComboBox,After the drop-down option is selected, reassign the drop-down list, and the selected data does not echo correctly 4 2022-08-18 2022-09-04
What is the best way to demonstrate Kafka at the Edge? 0 2021-07-19 2022-09-27
[module_variable_optional_attrs ] Terraform ignores changes in default section of the variable 6 2022-04-12 2022-09-19
Upgrade version 0.1.2, login appears / error type = none, status = 999 3 2021-07-15 2022-09-26
Refactor `diffViewList` to use `longestIncreasingSubsequence` 0 2021-09-07 2022-09-24