Hello,

When memory mapping a file with the MAP_PRIVATE flag, the modifications
(writes) only exist in memory and are not written back to the file.

According to the man pages, calling msync (3) on a such a mapping does NOT
writes the changes back:

"When the msync() function is called on MAP_PRIVATE mappings, any modified
data shall not be written to the underlying object and shall not cause
such data to be made visible to other processes"
https://linux.die.net/man/3/msync

So: is there a way to write the changes back to the file?

An obvious application is:
- mapping the file with MAP_PRIVATE
- make some modifications in memory only (fast) while keeping the original
version on disk (safe)
- at some point (when the user decides, and once the consistency of the
changes have been verified) writing the modifications to the disk

I'm pretty sure it exists some way or another, but I don't know how.

Le 07/04/2024 à 15:34, pehache a écrit :
> Hello,
>
> When memory mapping a file with the MAP_PRIVATE flag, the modifications (writes)
> only exist in memory and are not written back to the file.
>
> According to the man pages, calling msync (3) on a such a mapping does NOT
> writes the changes back:
>
> "When the msync() function is called on MAP_PRIVATE mappings, any modified data
> shall not be written to the underlying object and shall not cause such data to be
> made visible to other processes"
> https://linux.die.net/man/3/msync
>
> So: is there a way to write the changes back to the file?
>
> An obvious application is:
> - mapping the file with MAP_PRIVATE
> - make some modifications in memory only (fast) while keeping the original
> version on disk (safe)
> - at some point (when the user decides, and once the consistency of the changes
> have been verified) writing the modifications to the disk
>
> I'm pretty sure it exists some way or another, but I don't know how.

At the moment what I'm doing is something like:

========================================
fd = open("aaa",O_RDWR);
p = mmap ( NULL
, n
, PROT_READ | PROT_WRITE
, MAP_PRIVATE | MAP_NORESERVE
, fd
, 0 );

// writing to p; the changes exist only in memory

void* p2 = mmap( NULL
, n
, PROT_READ | PROT_WRITE
, MAP_SHARED | MAP_NORESERVE
, fd
, 0 );

memcpy(p2,p,n); // copying everything from p to p2
msync(p2,n);

// unmap/remap p so it's ready for new changes
munmap(p,n);
p = mmap ( NULL
, n
, PROT_READ | PROT_WRITE
, MAP_PRIVATE | MAP_NORESERVE
, fd
, 0 );

========================================

This works, but:

- the whole content is copied, not only the changed content
- is this code legal? Is there any potential conflict between the 2
mapping, with an undefined behavior?

On Sun, 07 Apr 2024 13:34:43 +0000, pehache wrote:

> Hello,

Hi, pehache

> When memory mapping a file with the MAP_PRIVATE flag, the modifications
> (writes) only exist in memory and are not written back to the file.
[snip]
> So: is there a way to write the changes back to the file?
[snip]

The comp.unix.programmer newsgroup will be of better help for this sort
of question (it's not really on-topic for comp.lang.c).

But, of the top of my head; with my limited understanding of the unix
mmap() kernel call, the only ways to "write the changes" are to either
1) mmap(MAP_SHARED) and modify the mapped area, or
2) write() the mapped area back to the file.

But, as I said, comp.unix.programmer will be of better help.

--
Lew Pitcher
"In Skills We Trust"

As advised, I'm copying this post here

Le 07/04/2024 à 17:18, pehache a écrit :
> Le 07/04/2024 à 15:34, pehache a écrit :
>> Hello,
>>
>> When memory mapping a file with the MAP_PRIVATE flag, the modifications (writes)
>> only exist in memory and are not written back to the file.
>>
>> According to the man pages, calling msync (3) on a such a mapping does NOT
>> writes the changes back:
>>
>> "When the msync() function is called on MAP_PRIVATE mappings, any modified data
>> shall not be written to the underlying object and shall not cause such data to be
>> made visible to other processes"
>> https://linux.die.net/man/3/msync
>>
>> So: is there a way to write the changes back to the file?
>>
>> An obvious application is:
>> - mapping the file with MAP_PRIVATE
>> - make some modifications in memory only (fast) while keeping the original
>> version on disk (safe)
>> - at some point (when the user decides, and once the consistency of the changes
>> have been verified) writing the modifications to the disk
>>
>> I'm pretty sure it exists some way or another, but I don't know how.
>
> At the moment what I'm doing is something like:
>
> ========================================
> fd = open("aaa",O_RDWR);
> p = mmap ( NULL
> , n
> , PROT_READ | PROT_WRITE
> , MAP_PRIVATE | MAP_NORESERVE
> , fd
> , 0 );
>
> // writing to p; the changes exist only in memory
>
> void* p2 = mmap( NULL
> , n
> , PROT_READ | PROT_WRITE
> , MAP_SHARED | MAP_NORESERVE
> , fd
> , 0 );
>
> memcpy(p2,p,n); // copying everything from p to p2
> msync(p2,n);
>
> // unmap/remap p so it's ready for new changes
> munmap(p,n);
> p = mmap ( NULL
> , n
> , PROT_READ | PROT_WRITE
> , MAP_PRIVATE | MAP_NORESERVE
> , fd
> , 0 );
>
> ========================================
>
> This works, but:
>
> - the whole content is copied, not only the changed content
> - is this code legal? Is there any potential conflict between the 2 mapping,
> with an undefined behavior?

Le 07/04/2024 à 15:49, Lew Pitcher a écrit :
> On Sun, 07 Apr 2024 13:34:43 +0000, pehache wrote:
>
>> Hello,
>
> Hi, pehache
>
>> When memory mapping a file with the MAP_PRIVATE flag, the modifications
>> (writes) only exist in memory and are not written back to the file.
> [snip]
>> So: is there a way to write the changes back to the file?
> [snip]
>
> The comp.unix.programmer newsgroup will be of better help for this sort
> of question (it's not really on-topic for comp.lang.c).
>
> But, of the top of my head; with my limited understanding of the unix
> mmap() kernel call, the only ways to "write the changes" are to either
> 1) mmap(MAP_SHARED) and modify the mapped area, or
> 2) write() the mapped area back to the file.

I'm currenly using the first option in my code. Either way, the whole
mapped area is written back, even if there were only a few changes. I was
hoping there could be another way to wrtite back only the modified pages.

> But, as I said, comp.unix.programmer will be of better help.

fu2

On Sun, 07 Apr 24 15:18:47 +0000, pehache wrote:

> void* p2 = mmap( NULL
> , n
> , PROT_READ | PROT_WRITE
> , MAP_SHARED | MAP_NORESERVE
> , fd
> , 0 );

Not easy to remember what the arguments mean. Try this:

void * p2 = mmap
(
/*addr =*/ NULL,
/*length =*/ n,
/*prot =*/ PROT_READ | PROT_WRITE,
/*flags =*/ MAP_SHARED | MAP_NORESERVE,
/*fd =*/ fd,
/*offset =*/ 0
);

On 2024-04-08, Lawrence D'Oliveiro <ldo@nz.invalid> wrote:
> On Sun, 07 Apr 24 15:18:47 +0000, pehache wrote:
>
>> void* p2 = mmap( NULL
>> , n
>> , PROT_READ | PROT_WRITE
>> , MAP_SHARED | MAP_NORESERVE
>> , fd
>> , 0 );
>
> Not easy to remember what the arguments mean. Try this:

Nobody is going to follow your grotesque, poorly considered coding
conventions.

>
> void * p2 = mmap
> (
> /*addr =*/ NULL,
^^^^ ^^^^ a null pointer is some kind of "addr"

> /*length =*/ n,
> /*prot =*/ PROT_READ | PROT_WRITE,
^^^^ ^^^^
| |
`---------|---- You know this
`---- from this.

> /*flags =*/ MAP_SHARED | MAP_NORESERVE,
^^^^
Tells you nothing. You know that preprocessor constants
combined together with | are "flags".

> /*fd =*/ fd,
^^ ^^
identical, tells you nothing.

> /*offset =*/ 0
> );

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On 08/04/2024 02:50, Lawrence D'Oliveiro wrote:
> On Sun, 07 Apr 24 15:18:47 +0000, pehache wrote:
>
>> void* p2 = mmap( NULL
>> , n
>> , PROT_READ | PROT_WRITE
>> , MAP_SHARED | MAP_NORESERVE
>> , fd
>> , 0 );
>
> Not easy to remember what the arguments mean. Try this:
>
> void * p2 = mmap
> (
> /*addr =*/ NULL,
> /*length =*/ n,
> /*prot =*/ PROT_READ | PROT_WRITE,
> /*flags =*/ MAP_SHARED | MAP_NORESERVE,
> /*fd =*/ fd,
> /*offset =*/ 0
> );

That's great but, how did you manage to figure out the meanings and
order of the parameters in order to be able to write those comments?

If you have 1000s of calls to such functions, will you have those
comments plastered everwhere?

In a language with proper named/keyword arguments, you don't need to
remember the order. You don't need to supply all the arguments (eg. addr
and offset in your example can have defaults, possibly those sets of
flags too).

If you get an argument name wrong, it will tell you. I suspect that if
you wrote this by mistake:

/*prot =*/ MAP_SHARED | MAP_NORESERVE,
/*flags =*/ PROT_READ | PROT_WRITE,

it would not be detected.

Le 07/04/2024 à 23:49, Keith Thompson a écrit :
> James Kuyper <jameskuyper@alumni.caltech.edu> writes:
>> On 4/7/24 15:09, pehache wrote:
>>> Le 07/04/2024 à 15:49, Lew Pitcher a écrit :
>> ...
>>>> But, as I said, comp.unix.programmer will be of better help.
>>>
>>> fu2
>>
>> Why does that advice annoy you? Why wouldn't you want to take your
>> question to the forum where people are best qualified to answer it?
>
> I believe "fu2" refers to this header line in the article you're
> replying to :
> Followup-To: comp.unix.programmer

That's correct.

>
> pehache: Apparently James thought "fu2" meant ... something else.

OK, sorry for the misunderstanding :( !

"fu2" is a routinely used abbreviation for "Followup-To" (and just that!)
on the fr.* hierarchy, and I thought it was kind of universal on all
hierarchies. Apparently it's not...

In article <PgZyW9oCgOxERNyTFwXvTAzcfig@jntp>,
pehache <pehache.7@gmail.com> wrote:
....
>OK, sorry for the misunderstanding :( !

Nothing to be sorry about.

>"fu2" is a routinely used abbreviation for "Followup-To" (and just that!)
>on the fr.* hierarchy, and I thought it was kind of universal on all
>hierarchies. Apparently it's not...

It is fine.

This poster (James whatever) has a long history of weirdly misunderstanding
things. I wouldn't worry about it.

--
Many people in the American South think that DJT is, and will be remembered
as, one of the best presidents in US history. They are absolutely correct.

He is currently at number 46 on the list. High praise, indeed!

On Mon, 8 Apr 2024 12:14:30 +0100, bart wrote:

> ... how did you manage to figure out the meanings and
> order of the parameters in order to be able to write those comments?

I read the docs.

> If you have 1000s of calls to such functions, will you have those
> comments plastered everwhere?

Yes.

> In a language with proper named/keyword arguments, you don't need to
> remember the order.

True. I take advantage of that--and defaults for omitted arguments--in
Python.

But there are people in this noisegroup who don’t like me mentioning that.