runsc: don't enable MADV_HUGEPAGE with direct compaction

[copybara-service]

runsc: don't enable MADV_HUGEPAGE with direct compaction

By default, private anonymous memory mappings in Linux use transparent
hugepages when available (/sys/kernel/mm/transparent_hugepage/enabled=always);
when hugepages are not available (typically due to fragmentation), such
mappings fall back to small pages instead
(/sys/kernel/mm/transparent_hugepage/defrag=madvise). See Linux's
Documentation/admin-guide/mm/transhuge.rst.

In gVisor, application private anonymous memory cannot generally be backed by
host private anonymous memory, since in many platforms, applications and the
sentry (which also needs to map application memory) run in different host
processes. Instead, application private anonymous memory is backed by a host
memfd, managed by pgalloc.MemoryFile. By default, Linux memfds never use
transparent hugepages
(/sys/kernel/mm/transparent_hugepage/shmem_enabled=never); thus
runsc/hostsettings sets shmem_enabled=madvise, allowing pgalloc to request
transparent hugepages using madvise(MADV_HUGEPAGE).

However, since the default value of /sys/kernel/mm/transparent_hugepage/defrag
is madvise, MADV_HUGEPAGE has the unintended side effect of enabling direct
compaction: when hugepages are not available, the host kernel will attempt to
form free hugepages by defragmenting small pages. This can be very expensive,
which is why (as described above) Linux doesn't do it by default.

Thus:

• When MADV_HUGEPAGE is required for application THP, only enable it if
  /sys/kernel/mm/transparent_hugepage/defrag specifies an operating mode that
  does not result in more direct compaction than a normal private anonymous
  mapping would.

• Adjust /sys/kernel/mm/transparent_hugepage/defrag accordingly in
  runsc/hostsettings.

FUTURE_COPYBARA_INTEGRATE_REVIEW=#11473 from Champ-Goblem:shim-add-cgroup-v2-metrics-support b602afb