[cmucl-help] maximum dynamic-space-size

[Carl Shapiro]

On Tue, Jan 5, 2016 at 7:08 PM, Tom Mitchell <mitch at niftyegg.com> wrote:

> Is there a way to protect from the Linux kernel  overcommit option?
> One real problem when overcommit is set to true is long running
> programs can run out of resources hours and days into a run.
> Compare malloc()  and calloc() and how errors are handled when
> system limits or system resources are exhausted.
>

This giant reservation is done for convenience.  Instead of performing a
single reservation, the garbage collector can reserve raw virtual memory in
a series of smaller reservations on an as-need basis.

Such a scheme would be a straightforward modification of the existing
garbage collector.  Additional reservations would be made when the pages
known to the "page table" are exhausted (not unlike sliding the break).
Any takers?

The value of overcommit is that a large long running job can
> do a fork(); exec() pair where the exec is a smallish process like
> print.answer.   The duplicated pages from the fork() and the associated
> bookkeeping cause no harm because copy on write never happens.
>

I believe that all of targets we care about provide a posix_spawn which
seems to be implemented as a vfork followed by an exec on FreeBSD, Linux,
and Solaris.  This avoids some of the costs associated with a copy-on-write
fork.

The notion of "maximum" is almost never correct on shared commons
> computing systems which is today's most common systems.
>
> *$*sysctl -a | grep overcom
> vm.overcommit_kbytes = 0
> vm.overcommit_memory = 0
> vm.overcommit_ratio = 50
>

Agreed.