I was learning about making user defined storage pools when
I came across an article that made me pause and wonder how
portable storage pools actually can be. In particular, I assumed
that the Size_In_Storage_Elements parameter in the Allocate
operation actually indicated the total number of storage elements
needed.

procedure Allocate(
Pool : in out Root_Storage_Pool;
Storage_Address : out Address;
Size_In_Storage_Elements : in Storage_Elements.Storage_Count;
Alignment : in Storage_Elements.Storage_Count) is abstract;

But after reading the following AdaCore article, my assumption is now
called into question:
https://blog.adacore.com/header-storage-pools

In particular, the blog there advocates for separately counting for
things like unconstrained array First/Last indices or the Prev/Next
pointers used for Controlled objects. Normally I would have assumed
that the Size_In_Storage_Elements parameter in Allocate would account
for that, but the blog clearly shows that it doesn't

So that seems to mean to make a storage pool, I have to make it
compiler specific or else risk someone creating a type like an
array and my allocation size and address values will be off.

Is it intended not to be able to do portable Storage Pools or am
I missing some Ada functionality that helps me out here. I
scanned through the list of attributes but none seem to give
any info about where the object's returned address is relative
to the top of the memory actually allocated for the object. I saw
the attribute Max_Size_In_Storage_Elements, but it doesn't seem
to guarantee to include things like the array indices and it still
doesn't solve the issue of knowing where the returned address
needs to be relative to the top of allocated memory.

I can easily use a generic to ensure that the types I care about
are portably made by the pool, but I can't prevent someone from
using my pool to create other objects that I hadn't accounted for.
Unless there is a way to restrict a pool from allocating objects
of other types?

Not sure what you are expecting. There is no requirement that objects are
allocated contigiously. Indeed, Janus/Ada will call Allocate as many times
as needed for each object; for instance, unconstrained arrays are in two
parts (descriptor and data area).

The only thing that you can assume in a portable library is that you get
called the same number of times and sizes/alignment for Allocate and
Deallocate; there's no assumptions about size or alignment that you can
make.

If you want to build a pool around some specific allocated size, then if it
needs to be portable, (A) you have to calculate the allocated size, and (B)
you have to have a mechanism for what to do if some other size is requested.
(Allocate a whole block for smaller sizes, fall back to built-in heap for
too large is what I usually do).

More likely, you'll build a pool for a particular implementation. Pools are
very low level by their nature, and useful ones are even more so (because
they are using target facilities to allocate memory, or need to assume
something about the allocations, or because they are doing icky things like
address math, or ...).

Randy.

Randy.

"Jere" <jhb.chat@gmail.com> wrote in message
news:e3c5c553-4a7f-408a-aaa7-60ec0b70202dn@googlegroups.com...
>I was learning about making user defined storage pools when
> I came across an article that made me pause and wonder how
> portable storage pools actually can be. In particular, I assumed
> that the Size_In_Storage_Elements parameter in the Allocate
> operation actually indicated the total number of storage elements
> needed.
>
> procedure Allocate(
> Pool : in out Root_Storage_Pool;
> Storage_Address : out Address;
> Size_In_Storage_Elements : in Storage_Elements.Storage_Count;
> Alignment : in Storage_Elements.Storage_Count) is abstract;
>
> But after reading the following AdaCore article, my assumption is now
> called into question:
> https://blog.adacore.com/header-storage-pools
>
> In particular, the blog there advocates for separately counting for
> things like unconstrained array First/Last indices or the Prev/Next
> pointers used for Controlled objects. Normally I would have assumed
> that the Size_In_Storage_Elements parameter in Allocate would account
> for that, but the blog clearly shows that it doesn't
>
> So that seems to mean to make a storage pool, I have to make it
> compiler specific or else risk someone creating a type like an
> array and my allocation size and address values will be off.
>
> Is it intended not to be able to do portable Storage Pools or am
> I missing some Ada functionality that helps me out here. I
> scanned through the list of attributes but none seem to give
> any info about where the object's returned address is relative
> to the top of the memory actually allocated for the object. I saw
> the attribute Max_Size_In_Storage_Elements, but it doesn't seem
> to guarantee to include things like the array indices and it still
> doesn't solve the issue of knowing where the returned address
> needs to be relative to the top of allocated memory.
>
> I can easily use a generic to ensure that the types I care about
> are portably made by the pool, but I can't prevent someone from
> using my pool to create other objects that I hadn't accounted for.
> Unless there is a way to restrict a pool from allocating objects
> of other types?

Le 13/09/2021 à 02:53, Jere a écrit :
> I was learning about making user defined storage pools when
> I came across an article that made me pause and wonder how
> portable storage pools actually can be. In particular, I assumed
> that the Size_In_Storage_Elements parameter in the Allocate
> operation actually indicated the total number of storage elements
> needed.
>
> procedure Allocate(
> Pool : in out Root_Storage_Pool;
> Storage_Address : out Address;
> Size_In_Storage_Elements : in Storage_Elements.Storage_Count;
> Alignment : in Storage_Elements.Storage_Count) is abstract;
>
> But after reading the following AdaCore article, my assumption is now
> called into question:
> https://blog.adacore.com/header-storage-pools
>
> In particular, the blog there advocates for separately counting for
> things like unconstrained array First/Last indices or the Prev/Next
> pointers used for Controlled objects. Normally I would have assumed
> that the Size_In_Storage_Elements parameter in Allocate would account
> for that, but the blog clearly shows that it doesn't
[...]

That blog shows a special use for Storage_Pools, where you allocate
/user/ data on top of the requested memory. When called by the compiler,
it is up to the compiler to compute how much memory is needed, and your
duty is to just allocate that.

--
J-P. Rosen
Adalog
2 rue du Docteur Lombard, 92441 Issy-les-Moulineaux CEDEX
Tel: +33 1 45 29 21 52
https://www.adalog.fr

On Monday, September 13, 2021 at 7:12:43 AM UTC-4, J-P. Rosen wrote:
> Le 13/09/2021 à 02:53, Jere a écrit :
> > I was learning about making user defined storage pools when
> > I came across an article that made me pause and wonder how
> > portable storage pools actually can be. In particular, I assumed
> > that the Size_In_Storage_Elements parameter in the Allocate
> > operation actually indicated the total number of storage elements
> > needed.
> >
> > procedure Allocate(
> > Pool : in out Root_Storage_Pool;
> > Storage_Address : out Address;
> > Size_In_Storage_Elements : in Storage_Elements.Storage_Count;
> > Alignment : in Storage_Elements.Storage_Count) is abstract;
> >
> > But after reading the following AdaCore article, my assumption is now
> > called into question:
> > https://blog.adacore.com/header-storage-pools
> >
> > In particular, the blog there advocates for separately counting for
> > things like unconstrained array First/Last indices or the Prev/Next
> > pointers used for Controlled objects. Normally I would have assumed
> > that the Size_In_Storage_Elements parameter in Allocate would account
> > for that, but the blog clearly shows that it doesn't
> [...]
>
> That blog shows a special use for Storage_Pools, where you allocate
> /user/ data on top of the requested memory. When called by the compiler,
> it is up to the compiler to compute how much memory is needed, and your
> duty is to just allocate that.
>
Yes, but if you look at that blog, they are allocating space for the /user/ data
and for the Next/Prev for controlled types and First/Last for unconstrained
arrays in addition to the size specified by allocate.

I agree I feel it is up to the compiler to provide the correct size to Allocate,
but the blog would indicate that GNAT does not (or did not..old blog..so
who knows?). Does the RM require that an implementation pass the full
amount of memory needed to Allocate when new is called?

On Monday, September 13, 2021 at 1:29:39 AM UTC-4, Randy Brukardt wrote:
> Not sure what you are expecting. There is no requirement that objects are
> allocated contigiously. Indeed, Janus/Ada will call Allocate as many times
> as needed for each object; for instance, unconstrained arrays are in two
> parts (descriptor and data area).
>
No expectations. Just questions. I wasn't concerned with whether the
allocated memory was contiguous or not, but whether an implementation
is required to supply the correct size of memory needed to allocate an object
or if it is allowed to pass a value to Size that is less than the amount of
memory actually needed. For example, the blog there indicates the
maintainer of the custom storage pool needs to account for First/Last
indexes of an unconstrained array separately instead of assuming that value is
included as part of the Size parameter's value.

If the Size parameter doesn't require that it includes space for First/Last
for unconstrained arrays or Prev/Next for controlled objects (assuming
that is even the implementation picked of course), then I'm not seeing
a way to write a custom storage pool that is portable because you need
to account for each implementation's "hidden" values that are not represented
in the Size parameter. For example if Janus calculated Size to have
both the size of the array and the size of First and Last but GNAT didn't
and my storage pool assumed the JANUS method, then if someone
used my storage pool with GNAT then it would access memory
from some other location potentially and erroneously.

> The only thing that you can assume in a portable library is that you get
> called the same number of times and sizes/alignment for Allocate and
> Deallocate; there's no assumptions about size or alignment that you can
> make.
So to be clear, you cannot assume that Size and Alignment are appropriate
for the actual object being allocated correct? Size could actually be
less than the actual amount of memory needed and the alignment may only
apply to part of the object being allocated, not the full object?

Is that correct? I'm asking because that is what the blog suggests with
the example it gave.

>
> If you want to build a pool around some specific allocated size, then if it
> needs to be portable, (A) you have to calculate the allocated size, and (B)
> you have to have a mechanism for what to do if some other size is requested.
> (Allocate a whole block for smaller sizes, fall back to built-in heap for
> too large is what I usually do).
>
Are there any good tricks to handle this? For example, if I design a
storage pool around constructing a particular type of object, what is
normally done to discourage another programmer from using the pool with
an entirely different type? Maybe raise an exception if the size isn't exact?
I'm not sure what else, unless maybe there is an Aspect/Attribute that
can be set to ensure only a specific type of object can be constructed.

HISTORY:
>
>
>
> "Jere" <> wrote in message
> news:e3c5c553-4a7f-408a...@googlegroups.com...
> >I was learning about making user defined storage pools when
> > I came across an article that made me pause and wonder how
> > portable storage pools actually can be. In particular, I assumed
> > that the Size_In_Storage_Elements parameter in the Allocate
> > operation actually indicated the total number of storage elements
> > needed.
> >
> > procedure Allocate(
> > Pool : in out Root_Storage_Pool;
> > Storage_Address : out Address;
> > Size_In_Storage_Elements : in Storage_Elements.Storage_Count;
> > Alignment : in Storage_Elements.Storage_Count) is abstract;
> >
> > But after reading the following AdaCore article, my assumption is now
> > called into question:
> > https://blog.adacore.com/header-storage-pools
> >
> > In particular, the blog there advocates for separately counting for
> > things like unconstrained array First/Last indices or the Prev/Next
> > pointers used for Controlled objects. Normally I would have assumed
> > that the Size_In_Storage_Elements parameter in Allocate would account
> > for that, but the blog clearly shows that it doesn't
> >
> > So that seems to mean to make a storage pool, I have to make it
> > compiler specific or else risk someone creating a type like an
> > array and my allocation size and address values will be off.
> >
> > Is it intended not to be able to do portable Storage Pools or am
> > I missing some Ada functionality that helps me out here. I
> > scanned through the list of attributes but none seem to give
> > any info about where the object's returned address is relative
> > to the top of the memory actually allocated for the object. I saw
> > the attribute Max_Size_In_Storage_Elements, but it doesn't seem
> > to guarantee to include things like the array indices and it still
> > doesn't solve the issue of knowing where the returned address
> > needs to be relative to the top of allocated memory.
> >
> > I can easily use a generic to ensure that the types I care about
> > are portably made by the pool, but I can't prevent someone from
> > using my pool to create other objects that I hadn't accounted for.
> > Unless there is a way to restrict a pool from allocating objects
> > of other types?

Le 14/09/2021 à 02:48, Jere a écrit :
>>> In particular, the blog there advocates for separately counting for
>>> things like unconstrained array First/Last indices or the Prev/Next
>>> pointers used for Controlled objects. Normally I would have assumed
>>> that the Size_In_Storage_Elements parameter in Allocate would account
>>> for that, but the blog clearly shows that it doesn't
>> [...]
>>
>> That blog shows a special use for Storage_Pools, where you allocate
>> /user/ data on top of the requested memory. When called by the compiler,
>> it is up to the compiler to compute how much memory is needed, and your
>> duty is to just allocate that.
>>
> Yes, but if you look at that blog, they are allocating space for the /user/ data
> and for the Next/Prev for controlled types and First/Last for unconstrained
> arrays in addition to the size specified by allocate.
>
> I agree I feel it is up to the compiler to provide the correct size to Allocate,
> but the blog would indicate that GNAT does not (or did not..old blog..so
> who knows?). Does the RM require that an implementation pass the full
> amount of memory needed to Allocate when new is called?
>

The RM says that an allocator allocates storage from the storage pool.
You could argue that it does not say "allocates all needed storage...",
but that would be a bit far fetched.

Anyway, a blog is not the proper place to get information from for that
kind of issue. Look at the Gnat documentation.

--
J-P. Rosen
Adalog
2 rue du Docteur Lombard, 92441 Issy-les-Moulineaux CEDEX
Tel: +33 1 45 29 21 52
https://www.adalog.fr

On 2021-09-14 02:48, Jere wrote:

> Yes, but if you look at that blog, they are allocating space for the /user/ data
> and for the Next/Prev for controlled types and First/Last for unconstrained
> arrays in addition to the size specified by allocate.

I do not understand your concern. The blog discusses how to add service
data to the objects allocated in the pool.

I use such pools extensively in Simple Components. E.g. linked lists are
implemented this way. The list links are allocated in front of list
elements which can be of any type, unconstrained arrays included.

The problem with unconstrained arrays is not that the bounds are not
allocated, they are, but the semantics of X'Address when applied to arrays.

A'Address is the address of the first array element, not of the array
object. For a pool designer it constitutes a problem of getting the
array object by address. This is what Emmanuel discusses in the blog.

[ The motivation behind Ada choice was probably to keep the semantics
implementation-independent. ]

Consider for example a list of String elements. When Allocate is called
with String, it returns the address of all String. But that is not the
address you would get if you applied 'Address. You have to add/subtract
some offset in order to get one from another.

In Simple Components this offset is determined at run-time for each
generic instance.

Of course, a proper solution would be fixing Ada by adding another
address attribute:

X'Object_Address

returning the first address of the object as allocated.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Le 14/09/2021 à 08:23, Dmitry A. Kazakov a écrit :
> Of course, a proper solution would be fixing Ada by adding another
> address attribute:
>
>    X'Object_Address
>
> returning the first address of the object as allocated.
But you cannot assume that the object is allocated as one big chunk.
Bounds can be allocated at a different place. What would be
X'Object_Address in that case?

--
J-P. Rosen
Adalog
2 rue du Docteur Lombard, 92441 Issy-les-Moulineaux CEDEX
Tel: +33 1 45 29 21 52
https://www.adalog.fr

On 2021-09-14 08:42, J-P. Rosen wrote:
> Le 14/09/2021 à 08:23, Dmitry A. Kazakov a écrit :
>> Of course, a proper solution would be fixing Ada by adding another
>> address attribute:
>>
>>     X'Object_Address
>>
>> returning the first address of the object as allocated.
> But you cannot assume that the object is allocated as one big chunk.
> Bounds can be allocated at a different place. What would be
> X'Object_Address in that case?

The object address, without bounds, same as X'Address.

What Allocate returns is not what A'Address tells. The compiler always
knows the difference, the programmer have to know it too. Nothing more.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

On Tuesday, September 14, 2021 at 2:48:16 AM UTC+2, Jere wrote:
>
> Yes, but if you look at that blog, they are allocating space for the /user/ data
> and for the Next/Prev for controlled types and First/Last for unconstrained
> arrays in addition to the size specified by allocate.
>

Yes, but if you look at that blog, they explain the default layout of fat pointers,
and the special value that need to be set on access types for the layout to
change. If you use such a GNAT-ism, your storage pool will also be bound
to GNAT...

ie:
"GNAT typically uses a "fat pointer" for this purpose: the access itself is in fact
a record of two pointers, one of which points to the bounds, the other points to
the data. This representation is not appropriate in the case of the header
storage pool, so we need to change the memory layout here."

and:
"we need to ensure that the bounds for unconstrained arrays are stored next to
the element, not in a separate memory block, to improve performance. This is
done by setting the Size attribute on the type. When we set this size to that of
a standard pointer, GNAT automatically changes the layout,"

--
~egilhh

On Tuesday, September 14, 2021 at 6:54:54 AM UTC-4, ehh wrote:
> On Tuesday, September 14, 2021 at 2:48:16 AM UTC+2, Jere wrote:
> >
> > Yes, but if you look at that blog, they are allocating space for the /user/ data
> > and for the Next/Prev for controlled types and First/Last for unconstrained
> > arrays in addition to the size specified by allocate.
> >
> Yes, but if you look at that blog, they explain the default layout of fat pointers,
> and the special value that need to be set on access types for the layout to
> change. If you use such a GNAT-ism, your storage pool will also be bound
> to GNAT...
>
>
> ie:
> "GNAT typically uses a "fat pointer" for this purpose: the access itself is in fact
> a record of two pointers, one of which points to the bounds, the other points to
> the data. This representation is not appropriate in the case of the header
> storage pool, so we need to change the memory layout here."
>
> and:
> "we need to ensure that the bounds for unconstrained arrays are stored next to
> the element, not in a separate memory block, to improve performance. This is
> done by setting the Size attribute on the type. When we set this size to that of
> a standard pointer, GNAT automatically changes the layout,"
>

What I am seeing in the blog is if I *do not* use the GNAT ism in my storage pool
and assume that the Size parameter indicates the full size needed, then if someone
uses my pool with a GNAT compiler it would erroneously access memory.

You can clearly see the calculation in the blog is:
Aligned_Size : constant Storage_Count := -- bytes
Size + Header_Allocation + Extra_Offset;

Where Size is specified by Allocate, Header_Allocation is the user supplied fields
of the storage pool, and ***Extra_Offset*** is a separate size value that accounts
for First/Last or Previous/Next

Finalization_Master_Size : constant Storage_Count :=
2 * Standard'Address_Size; -- for Previous and Next
Extra_Offset : constant Storage_Count :=
(Element_Type'Descriptor_Size + Finalization_Master_Size)
/ Storage_Unit; -- convert from bits to bytes

The blog quotes:
"""The size for the bounds is given by the attribute Descriptor_Size. For most types,
the value of this attribute is 0. However, in the case of unconstrained array types, it
is the size of two integers"""

These values are specified separate from the Size parameter and added to it in the
Allocate function shown in the blog there.

If I were to write my own storage pool and I didn't do the Extra_Offset calculation and just
assumed the compiler would call Allocate with the correct Size value for everything,
the logic in that blog then would indicate that GNAT would assume I did the Extra_Offset
calculation anyways and it could access those fields even if I didn't explicitly allocate
them separately as GNAT does. In that case the fields would be part of some other object's
memory or random memory.

My original question is if the intended premise in Ada is that Storage pools aren't expected to be
created to be portable due to just being low level because compilers can just assume
you will know to allocate hidden fields in addition to what the Size parameter specifies.

Does that help clarify what I'm confused on? Sorry, I am not great with wording.

On Tuesday, September 14, 2021 at 2:23:15 AM UTC-4, Dmitry A. Kazakov wrote:
> On 2021-09-14 02:48, Jere wrote:
>
> > Yes, but if you look at that blog, they are allocating space for the /user/ data
> > and for the Next/Prev for controlled types and First/Last for unconstrained
> > arrays in addition to the size specified by allocate.
> I do not understand your concern. The blog discusses how to add service
> data to the objects allocated in the pool.
>
I tried to better articulate my concern in my response to egilhh if you want
to take a quick look at that and see if it clarifies better.

> I use such pools extensively in Simple Components. E.g. linked lists are
> implemented this way. The list links are allocated in front of list
> elements which can be of any type, unconstrained arrays included.
>
The blog I saw was old, so it is completely possible it no longer is
true that GNAT does what the blog suggests. I'll take a look at your
storage pools and see how they handle things like this.

On Tuesday, September 14, 2021 at 2:08:49 AM UTC-4, J-P. Rosen wrote:
> Le 14/09/2021 à 02:48, Jere a écrit :
> >>> In particular, the blog there advocates for separately counting for
> >>> things like unconstrained array First/Last indices or the Prev/Next
> >>> pointers used for Controlled objects. Normally I would have assumed
> >>> that the Size_In_Storage_Elements parameter in Allocate would account
> >>> for that, but the blog clearly shows that it doesn't
> >> [...]
> >>
> >> That blog shows a special use for Storage_Pools, where you allocate
> >> /user/ data on top of the requested memory. When called by the compiler,
> >> it is up to the compiler to compute how much memory is needed, and your
> >> duty is to just allocate that.
> >>
> > Yes, but if you look at that blog, they are allocating space for the /user/ data
> > and for the Next/Prev for controlled types and First/Last for unconstrained
> > arrays in addition to the size specified by allocate.
> >
> > I agree I feel it is up to the compiler to provide the correct size to Allocate,
> > but the blog would indicate that GNAT does not (or did not..old blog..so
> > who knows?). Does the RM require that an implementation pass the full
> > amount of memory needed to Allocate when new is called?
> >
> The RM says that an allocator allocates storage from the storage pool.
> You could argue that it does not say "allocates all needed storage...",
> but that would be a bit far fetched.
I agree, but the blog made reconsider how far fetched it was.

>
> Anyway, a blog is not the proper place to get information from for that
> kind of issue. Look at the Gnat documentation.
> --
I'll take a look at the GNAT docs to see (and of course that blog is old,
so GNAT may not do this anymore anyways), but I mainly asking in the
frame of what Ada allows and/or expects. I'd like to be able to allocate
storage simply without worrying how the compiler does it under the hood
and just assume that any calls to Allocate will ask for the full amount of
memory.

Am I correct to assume that Ada doesn't provide any language means
to restrict what types a pool can make objects of. The times that I have
wanted to make a pool are generally for specific types and it is often
simpler to design them if I can assume only those types are being generated

Thanks for the response. I'm sorry for all the questions. That's how I
learn and I realize it isn't a popular way to learn in the community, but
I have always learned very differently than most.

On 2021-09-15 02:21, Jere wrote:
> On Tuesday, September 14, 2021 at 2:23:15 AM UTC-4, Dmitry A. Kazakov wrote:
>> On 2021-09-14 02:48, Jere wrote:
>>
>>> Yes, but if you look at that blog, they are allocating space for the /user/ data
>>> and for the Next/Prev for controlled types and First/Last for unconstrained
>>> arrays in addition to the size specified by allocate.
>> I do not understand your concern. The blog discusses how to add service
>> data to the objects allocated in the pool.
>>
> I tried to better articulate my concern in my response to egilhh if you want
> to take a quick look at that and see if it clarifies better.

Not really. It seems that you are under impression that Allocate must
allocate more size than its Size parameter asks. The answer is no,
unless *you* wanted to add something to each allocated object.

>> I use such pools extensively in Simple Components. E.g. linked lists are
>> implemented this way. The list links are allocated in front of list
>> elements which can be of any type, unconstrained arrays included.
>>
> The blog I saw was old, so it is completely possible it no longer is
> true that GNAT does what the blog suggests. I'll take a look at your
> storage pools and see how they handle things like this.

I calculate the offset at run time. I keep the list of recently
allocated blocks. The first time the address is asked from an element,
that is, the first time an element allocated in the pool becomes known
to the pool, its address is searched through the list of blocks. The
minimal difference between the block address and the element address
(X'Address) is the offset.

This is more portable than GNAT-specific attribute Descriptor_Size.

Again, if attributes to be added, then it should be the object address
as allocated. The compiler always knows the proper address because this
address is passed to Free, not X'Address!

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Jere <jhb.chat@gmail.com> writes:

> Thanks for the response. I'm sorry for all the questions. That's how
> I learn and I realize it isn't a popular way to learn in the
> community, but I have always learned very differently than most.

Seems to me you ask interesting questions which generate enlightening
responses!

Jere <jhb.chat@gmail.com> writes:

> But after reading the following AdaCore article, my assumption is now
> called into question:
> https://blog.adacore.com/header-storage-pools
>
> In particular, the blog there advocates for separately counting for
> things like unconstrained array First/Last indices or the Prev/Next
> pointers used for Controlled objects. Normally I would have assumed
> that the Size_In_Storage_Elements parameter in Allocate would account
> for that, but the blog clearly shows that it doesn't

Well, I may well have missed the point somewhere, and maybe things have
changed since 2015, but as far as I can see, with FSF GCC 11.1.0, the
technique described in the blog is completely unnecessary.

To save having to recompile the runtime with debug symbols, I wrote a
tiny pool which delegates to GNAT's
System.Pool_Global.Global_Pool_Object (the default pool), e.g.

overriding procedure Allocate
(Pool : in out My_Pool.Pool;
Storage_Address : out Address;
Size_In_Storage_Elements : in Storage_Elements.Storage_Count;
Alignment : in Storage_Elements.Storage_Count)
is
pragma Unreferenced (Pool);
begin
Global_Pool_Object.Allocate
(Address => Storage_Address,
Storage_Size => Size_In_Storage_Elements,
Alignment => Alignment);
end Allocate;

and I find with

Pool : My_Pool.Pool;

type C is new Ada.Finalization.Controlled with null record;
type Cs is array (Natural range <>) of C;
type Csp is access Cs with Storage_Pool => Pool;
procedure Free is new Ada.Unchecked_Deallocation (Cs, Csp);
Pcs : Csp;

begin

Pcs := new Cs (0 .. 5);
Free (Pcs);

that

* the alignment requested is 8 (was 4 for an array of Boolean);
* the size requested is 72, which is 24 bytes more than required for the
6 minimal POs;
* the value returned by Allocate is 24 bytes more than the address of
the array object Pcs (which is the same as that of Pcs(0));
* the value passed to Deallocate is the same as that returned by
Allocate.

I think it's more than likely (!) that the extra allocation of 24 bytes
is made up of 2 pointers at 8 bytes each, used to implement the
finalization chain, and two integers at 4 bytes each, holding the array
bounds.

So I'd say that to create a pool with extra header information, you'd
need to allocate space for your header + padding to ensure that the
compiler's object is properly aligned + the compiler-requested size,
aligned to the max of your header's alignment and the compiler-requested
alignment.

Mind, I don't quite see how to actually access the header info for a
particular allocation ...

Simon Wright <simon@pushface.org> writes:

> Well, I may well have missed the point somewhere, and maybe things
> have changed since 2015, but as far as I can see, with FSF GCC 11.1.0,
> the technique described in the blog is completely unnecessary.

Looking at the current implementation in [1] it seems that I was right
(I don't quite follow the complication of Typed.Header_Of at line 114;
goes to show that if you're writing the compiler you can add attributes
whenever it suits you, even if IMO they weren't actually needed)

[1] https://github.com/AdaCore/gnatcoll-core/blob/master/src/gnatcoll-storage_pools-headers.adb

On 2021-09-15 18:43, Simon Wright wrote:

> To save having to recompile the runtime with debug symbols, I wrote a
> tiny pool which delegates to GNAT's
> System.Pool_Global.Global_Pool_Object (the default pool), e.g.
>
> overriding procedure Allocate
> (Pool : in out My_Pool.Pool;
> Storage_Address : out Address;
> Size_In_Storage_Elements : in Storage_Elements.Storage_Count;
> Alignment : in Storage_Elements.Storage_Count)
> is
> pragma Unreferenced (Pool);
> begin
> Global_Pool_Object.Allocate
> (Address => Storage_Address,
> Storage_Size => Size_In_Storage_Elements,
> Alignment => Alignment);
> end Allocate;
>
> and I find with
>
> Pool : My_Pool.Pool;
>
> type C is new Ada.Finalization.Controlled with null record;
> type Cs is array (Natural range <>) of C;
> type Csp is access Cs with Storage_Pool => Pool;

Now define and implement the following function:

function Get_Allocation_Time (Pointer : Csp) return Time;

The function returns the time when the pointed object was allocated in
the pool.

[...]

> Mind, I don't quite see how to actually access the header info for a
> particular allocation ...

By subtracting a fixed offset from Pointer.all'Address. The offset is to
be determined, because X'Address lies.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

"Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> writes:

>> Mind, I don't quite see how to actually access the header info for a
>> particular allocation ...
>
> By subtracting a fixed offset from Pointer.all'Address. The offset is
> to be determined, because X'Address lies.

Obvs.

But my main point was, the blog which was Jere's original problem is in
fact (now?) wrong.

Anyway, the Gnatcoll solution is here ...
https://github.com/AdaCore/gnatcoll-core/blob/master/src/gnatcoll-storage_pools-headers.adb#L114

I am the original implementor of GNATCOLL.Storage_Pools.Headers, and I have been silent in this discussion because I must admit I forgot a lot of the details... To be sure, we did not add new attributes just for the sake of GNATCOLL, those existed previously so likely had already found other uses.

As has been mentioned several time in the discussion, the compiler is already passing the full size it needs to Allocate, and the storage pool only needs to allocate that exact amount in general. This applies for the usual kinds of storage pools, which would for instance preallocate a pool for objects of fixed sizes, or add stronger alignment requirements.

In the case of the GNATCOLL headers pool, we need to allocate more because the user wants to store extra data. For that data, we are left on our own to find the number of bytes we need, which is part of the computation we do: we of course need the number of bytes for the header's object_size, but also perhaps some extra bytes that are not returned by that object_size in particular for controlled types and arrays.
Note again that those additional bytes are for the header type, not for the type the user is allocating (for which, again, the compiler already passes the number of bytes it needs).

The next difficulty is then to convert from the object's 'Address back to your extra header data. This is when you need to know the size of the prefix added by the compiler (array bounds, tag,...) to skip them and then take into account the alignment, and finally the size of your header.
Dmitry's suggested exercice (storing the timestamp of the allocation) seems like a useful one indeed. It would be nice indeed if GNATCOLL's code was too complicated, but I am afraid this isn't the case. We had used those pools to implement reference counting for various complex types, and ended up with that complexity.

AdaCore (Olivier Hainque) has made a change to the implementation since the blog was published (https://github.com/AdaCore/gnatcoll-core/commits/master/src/gnatcoll-storage_pools-headers.adb), so I got some details wrong in the initial implementation apparently.

Emmanuel

On 2021-09-15 22:40, Simon Wright wrote:

> But my main point was, the blog which was Jere's original problem is in
> fact (now?) wrong.

Right, when nothing to be added to the allocated object, there is
nothing to worry about.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

On Thursday, September 16, 2021 at 3:13:00 AM UTC-4, Emmanuel wrote:
> In the case of the GNATCOLL headers pool, we need to allocate more because the user wants to store extra data. For that data, we are left on our own to find the number of bytes we need, which is part of the computation we do: we of course need the number of bytes for the header's object_size, but also perhaps some extra bytes that are not returned by that object_size in particular for controlled types and arrays.
> Note again that those additional bytes are for the header type, not for the type the user is allocating (for which, again, the compiler already passes the number of bytes it needs).
>
> <SNIPEED>
> Emmanuel

Thanks for the response Emmanuel. That clears it up for me. I think the confusion for me
came from the terminology used then. In the blog, that extra space for First/Last and Prev/Next
was mentioned as if it were for the element, which I mistook was the user's object being allocated
and not the header portion. I didn't catch that as the generic formal's name, so that is my mistake.
I guess in my head, I would have expected the formal name to be Header_Type or similar so I
misread it in my haste.

I appreciate the clarity and apologize if I caused too much of a stir. I was asking the question
because I didn't understand, so I hope you don't think too poorly of me for it, despite my mistake.

On Wednesday, September 15, 2021 at 3:01:52 AM UTC-4, Simon Wright wrote:
> Jere <> writes:
>
> > Thanks for the response. I'm sorry for all the questions. That's how
> > I learn and I realize it isn't a popular way to learn in the
> > community, but I have always learned very differently than most.
> Seems to me you ask interesting questions which generate enlightening
> responses!
Thanks! though in this case, my question was ill formed after I missed a detail
in the blog, so the mistake is on me. I will say I hold back some questions
as it is very intimidating to ask on C.L.A. I mean the first response led off
with "Not sure what you are expecting" so it is hard to know how to formulate
a good question as I always seem to get some harsh responses (which I am
sure is because I asked the question poorly). I'm unfortunately a very visual
person and words are not my forte and I feel like when I ask questions about
the boundaries of the language I manage to put folks on the defensive. I
don't dislike Ada at all, it is my favorite language, but I think it is hard to
craft questions on some topics without putting forth the impression that
I don't like it, at least with my limited ability to word craft.

> Thanks for the response Emmanuel. That clears it up for me. I think the confusion for me
> came from the terminology used then. In the blog, that extra space for First/Last and Prev/Next
> was mentioned as if it were for the element, which I mistook was the user's object being allocated
> and not the header portion. I didn't catch that as the generic formal's name, so that is my mistake.
> I guess in my head, I would have expected the formal name to be Header_Type or similar so I
> misread it in my haste.

Sure, and maybe the blog was not as readable as it should have been. Unfortunately I no longer have
any possibility to amend it, so I guess we'll live with it !

Thanks for the questions

Emmanuel

Jere <jhb.chat@gmail.com> writes:

> On Thursday, September 16, 2021 at 3:13:00 AM UTC-4, Emmanuel wrote:
>> In the case of the GNATCOLL headers pool, we need to allocate more
>> because the user wants to store extra data. For that data, we are
>> left on our own to find the number of bytes we need, which is part of
>> the computation we do: we of course need the number of bytes for the
>> header's object_size, but also perhaps some extra bytes that are not
>> returned by that object_size in particular for controlled types and
>> arrays.
>> Note again that those additional bytes are for the header type, not
>> for the type the user is allocating (for which, again, the compiler
>> already passes the number of bytes it needs).

> Thanks for the response Emmanuel. That clears it up for me. I think
> the confusion for me came from the terminology used then. In the
> blog, that extra space for First/Last and Prev/Next was mentioned as
> if it were for the element, which I mistook was the user's object
> being allocated and not the header portion. I didn't catch that as
> the generic formal's name, so that is my mistake.

Given this diagram from the blog, you can hardly be blamed (says a
person who leapt to the same conclusion):
+--------+-------+------+----------+------+---------+
| Header | First | Last | Previous | Next | Element |
+--------+-------+------+----------+------+---------+