On Thursday, August 18, 2022 at 1:49:27 PM UTC-5, EricP wrote:
> Terje Mathisen wrote:
> >
> > diff = next_event_count - current_tick_count;
> > if ((signed) diff <= 0) fire();
> Unlike the system clock, the period counter resets to zero at boot.
<
> With a 1 ns tick a 64 bit counter wraps after 584 years of
> continuous up-time so it wouldn't need any extra handling.
<
At least the person who turned the system on won't have to listen
to the complaints of bad-SW when the timer does roll over.
>
> However if the counter or trigger is smaller, say 32 or 48 bits,
> or the ticks smaller, say 1 ps, then the counter wrap generates
> an interrupt that increments an epoch number in memory.
<
So, in effect, you are arguing that the timer resolution should
be better than 1ns and worse than 1ps. 1ps rolls a 64-bit counter
over every ½ year, 10ps every 5.8 years, 100 ps every 58 years.
>
> >
> The original spec also had design options like whether the counter
> or comparators were 32 or 64 bits, and if 64 bits are they read and
> written atomically or as two separate 32 bit values.
<
In this day and age, why would anyone want something smaller than
a 64-bit counter ???
<
> Software had to take all these variations into account.
>
> One of the interactions the Linux folks seemed concerned
> about is that an interrupt is generated when the counter wraps,
> and this can interact with power state management.
> A 32 bit counter with 1 ns resolution tick would overflow every
> 4 seconds and wake up the processor out of power down.
> Another reason to require 64 bit timer components.
<
I repeat::
In this day and age, why would anyone want something smaller than
a 64-bit counter ???

On Thursday, August 18, 2022 at 2:19:12 PM UTC-5, Stefan Monnier wrote:
> > However if the counter or trigger is smaller, say 32 or 48 bits,
> > or the ticks smaller, say 1 ps, then the counter wrap generates
> > an interrupt that increments an epoch number in memory.
> You'll want to make sure this epoch-interrupt is triggered often enough
> to be tested sufficiently to be reliable.
<
Initialize the counter value so that it rolls over within the current day.
>
>
> Stefan

Stephen Fuld wrote:
> On 8/18/2022 12:18 AM, Terje Mathisen wrote:
>> MitchAlsup wrote:
>>> On Wednesday, August 17, 2022 at 9:14:55 AM UTC-5, EricP wrote:
>>>> Thomas Koenig wrote:
>>>>> robf...@gmail.com <robf...@gmail.com> schrieb:
>>>>>
>>>>>> 30+ comparators per core (300 comparators?) might allow every
>>>>>> process to have its own comparator. Which would not need to be
>>>>>> managed by the OS.
>>>>>
>>>>> Hmm... do you want to restrict the core to 300 processes? Hard
>>>>> limits have a habit of coming around and biting the user when it
>>>>> least expected (for example when an attacker or a buggy process
>>>>> does a few forks() and uses up all of them).
>>>>>
>>>>> And you would need the OS anyway to schedule access to them so
>>>>> so it is guaranteed that no two processes use the same timer.
>>> <
>>>> You don't need to dedicate a high resolution timer to a process.
>>>> That wouldn't do much good as processes can have many timers
>>>> and software within a single process won't know about other timers.
>>>>
>>>> Each one-shot HW hr-comparator manages a queue of requests.
>>> <
>>> Yes, obviously...
>>> <
>>>> The queue can be a linked list or binary tree in ascending order
>>>> of absolute timeout hr-tick time. The earliest timeout timestamp
>>>> is written into the trigger register, or 0 to disable the trigger
>>>> if the queue is empty.
>>> <
>>> What does SW do in managing this list of event-times when one
>>> event fires, and the next event is 1 tick farther down in time ?
>>> It seems to me that the only way for SW to manage such a queue
>>> would be to have large timer-quanta.
>>
>> I think that there are only two ways it can work:
>>
>> a) Make the comparator >= so that when the next value is written, it
>> immediately causes a timeout since the tick count has passed. This
>> does require that comparator to separate between a recent rollower and
>> a long time to go, typically using 2^63 as the boundary:
>>
>>  Â  diff = next_event_count - current_tick_count;
>>  Â  if ((signed) diff <= 0) fire();
>>
>> b) Do the same in SW with a polled approach:
>>
>> When a event count triggers, the handler dispatches it, then grabs the
>> next count in the queue and inserts it before immediately reading the
>> current count to check if the time has already passed, if so, handle
>> it and repeat. The big problem here is of course that there will
>> always be a tiny window between writing the next comparator value and
>> reading the current count to check for an overrun. Fixing that adds a
>> lot of hair, you either need the (HW) ability to get the current count
>> at the same time as you write the comparator, or some kind of status
>> register to report on any currently pending timeout.
>>
>> It is very easy to see why the hw people would prefer a plain ==
>> comparator, but it does lead to even more issues in software. :-(
>
> I have seen timers that append a few bit field to the timer.  This field
> is returned as the low end of the value when one reads the timer.  The
> contents of this field are incremented each time the timer is read, and
> reset to zero each time the timer ticks.  The result guarantees a unique
> value returned, but does give a value where the bits below the specified
> minimum accuracy of the timer may not be accurate.
>
> For example, suppose you have a timer granularity of 1ns, and four bits
> in the new field.  Someone reading the timer would see a value specified
> in 16ths of a ns, but only the bits at or above 1 ns would be accurate.
> Thus someone reading the counter will usually get an accurate value (the
> extra bits zero), but occasionally it will be off by a small number of
> 16ths of a nanosecond.
>
> I thought this was an interesting tradeoff.

Very interesting, and possibly cheaper than just having a timer which
increments at least as fast as it can be read. I.e. RDTSC these days
gets you a cpu-fixed "base frequency" somewhere in the 2-5 GHz region,
but it takes multiple ns to read it. Here the issue is more related to
keeping that timer in sync across all the cores and possibly also across
cpus on a system with several.

A common bus-level counter typically incurs far more latency than RDTSC
and similar on-chip counters.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

robf...@gmail.com wrote:
> On Thursday, August 18, 2022 at 3:18:32 AM UTC-4, Terje Mathisen wrote:
>> MitchAlsup wrote:
>>> On Wednesday, August 17, 2022 at 9:14:55 AM UTC-5, EricP wrote:
>>>> Thomas Koenig wrote:
>>>>> robf...@gmail.com <robf...@gmail.com> schrieb:
>>>>>
>>>>>> 30+ comparators per core (300 comparators?) might allow every
>>>>>> process to have its own comparator. Which would not need to be
>>>>>> managed by the OS.
>>>>>
>>>>> Hmm... do you want to restrict the core to 300 processes? Hard
>>>>> limits have a habit of coming around and biting the user when it
>>>>> least expected (for example when an attacker or a buggy process
>>>>> does a few forks() and uses up all of them).
>>>>>
>>>>> And you would need the OS anyway to schedule access to them so
>>>>> so it is guaranteed that no two processes use the same timer.
>
> With enough timers the process ID could be used to determine which
> timer to use, so the OS does not need to schedule one. There are only
> about 250 process running on my Windows machine (3,000 threads).
> I think a timer component is not very large so there could be a lot of
> them.
>
>>> <
>>>> You don't need to dedicate a high resolution timer to a process.
>>>> That wouldn't do much good as processes can have many timers
>>>> and software within a single process won't know about other timers.
>>>>
>>>> Each one-shot HW hr-comparator manages a queue of requests.
>>> <
>>> Yes, obviously...
>>> <
>>>> The queue can be a linked list or binary tree in ascending order
>>>> of absolute timeout hr-tick time. The earliest timeout timestamp
>>>> is written into the trigger register, or 0 to disable the trigger
>>>> if the queue is empty.
>>> <
>>> What does SW do in managing this list of event-times when one
>>> event fires, and the next event is 1 tick farther down in time ?
>>> It seems to me that the only way for SW to manage such a queue
>>> would be to have large timer-quanta.
>> I think that there are only two ways it can work:
>>
>> a) Make the comparator >= so that when the next value is written, it
>> immediately causes a timeout since the tick count has passed. This does
>> require that comparator to separate between a recent rollower and a long
>> time to go, typically using 2^63 as the boundary:
>>
>> diff = next_event_count - current_tick_count;
>> if ((signed) diff <= 0) fire();
>>
>> b) Do the same in SW with a polled approach:
>>
>> When a event count triggers, the handler dispatches it, then grabs the
>> next count in the queue and inserts it before immediately reading the
>> current count to check if the time has already passed, if so, handle it
>> and repeat. The big problem here is of course that there will always be
>> a tiny window between writing the next comparator value and reading the
>> current count to check for an overrun. Fixing that adds a lot of hair,
>> you either need the (HW) ability to get the current count at the same
>> time as you write the comparator, or some kind of status register to
>> report on any currently pending timeout.
>>
>> It is very easy to see why the hw people would prefer a plain ==
>> comparator, but it does lead to even more issues in software. :-(
>> Terje
>>
>> --
>> - <Terje.Mathisen at tmsw.no>
>> "almost all programming can be viewed as an exercise in caching"
>
> How about writing the increment amount to the comparator instead
> of the absolute value? The current compare value would not need to be
> read then. The store operation would just automatically add to the current
> count.

That is actually very nice!

It puts all the slow/hard to check logic inside the write path where it
is trivial to determine if the new value will be less than the current
counts, and if so, latch a new interrupt to be triggered immeditely.

This way the on-the-fly trigger can stay a pure == which is of course
significantly less costly than >=.
>
> What about having two comparator types, one for high precision timing
> that works down to ns range and a second type that works on the range
> shifted over a number of bits, eight for example, to give a range of value
> that can be used to time events into the years? And then use a 72-bit
> counter.

I don't think this is needed in HW, it can easily be synthesized by the
driver.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

EricP wrote:
> Terje Mathisen wrote:
>> MitchAlsup wrote:
>>> On Wednesday, August 17, 2022 at 9:14:55 AM UTC-5, EricP wrote:
>>>> Thomas Koenig wrote:
>>>>> robf...@gmail.com <robf...@gmail.com> schrieb:
>>>>>
>>>>>> 30+ comparators per core (300 comparators?) might allow every
>>>>>> process to have its own comparator. Which would not need to be
>>>>>> managed by the OS.
>>>>>
>>>>> Hmm... do you want to restrict the core to 300 processes? Hard
>>>>> limits have a habit of coming around and biting the user when it
>>>>> least expected (for example when an attacker or a buggy process
>>>>> does a few forks() and uses up all of them).
>>>>>
>>>>> And you would need the OS anyway to schedule access to them so
>>>>> so it is guaranteed that no two processes use the same timer.
>>> <
>>>> You don't need to dedicate a high resolution timer to a process.
>>>> That wouldn't do much good as processes can have many timers
>>>> and software within a single process won't know about other timers.
>>>>
>>>> Each one-shot HW hr-comparator manages a queue of requests.
>>> <
>>> Yes, obviously...
>>> <
>>>> The queue can be a linked list or binary tree in ascending order
>>>> of absolute timeout hr-tick time. The earliest timeout timestamp
>>>> is written into the trigger register, or 0 to disable the trigger
>>>> if the queue is empty.
>>> <
>>> What does SW do in managing this list of event-times when one
>>> event fires, and the next event is 1 tick farther down in time ?
>>> It seems to me that the only way for SW to manage such a queue
>>> would be to have large timer-quanta.
>>
>> I think that there are only two ways it can work:
>>
>> a) Make the comparator >= so that when the next value is written, it
>> immediately causes a timeout since the tick count has passed. This
>> does require that comparator to separate between a recent rollower and
>> a long time to go, typically using 2^63 as the boundary:
>>
>>   diff = next_event_count - current_tick_count;
>>   if ((signed) diff <= 0) fire();
>
> Unlike the system clock, the period counter resets to zero at boot.
> With a 1 ns tick a 64 bit counter wraps after 584 years of
> continuous up-time so it wouldn't need any extra handling.

that is a very good, and probably sufficient, reason for making such a
counter 64-bit only. If you need the ability to work in a 32-bit
environment, then reading or writing the lower 32 bits would have to
latch the upper part as well, holding it until the next instruction is done.
>
> However if the counter or trigger is smaller, say 32 or 48 bits,
> or the ticks smaller, say 1 ps, then the counter wrap generates
> an interrupt that increments an epoch number in memory.
>
> The trigger register for the youngest timer at the head of the queue
> is only set when the current epoch number >= that timers' epoch.
> Otherwise the trigger is disabled.

OK.

>
>> b) Do the same in SW with a polled approach:
>>
>> When a event count triggers, the handler dispatches it, then grabs the
>> next count in the queue and inserts it before immediately reading the
>> current count to check if the time has already passed, if so, handle
>> it and repeat. The big problem here is of course that there will
>> always be a tiny window between writing the next comparator value and
>> reading the current count to check for an overrun. Fixing that adds a
>> lot of hair, you either need the (HW) ability to get the current count
>> at the same time as you write the comparator, or some kind of status
>> register to report on any currently pending timeout.
>>
>> It is very easy to see why the hw people would prefer a plain ==
>> comparator, but it does lead to even more issues in software. :-(
>
> The original spec also had design options like whether the counter
> or comparators were 32 or 64 bits, and if 64 bits are they read and
> written atomically or as two separate 32 bit values.
> Software had to take all these variations into account.
>
> One of the interactions the Linux folks seemed concerned
> about is that an interrupt is generated when the counter wraps,
> and this can interact with power state management.
> A 32 bit counter with 1 ns resolution tick would overflow every
> 4 seconds and wake up the processor out of power down.
> Another reason to require 64 bit timer components.

I agree 100%.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

On Friday, August 19, 2022 at 10:25:13 AM UTC+3, Terje Mathisen wrote:
> EricP wrote:
> > Terje Mathisen wrote:
> >> MitchAlsup wrote:
> >>> On Wednesday, August 17, 2022 at 9:14:55 AM UTC-5, EricP wrote:
> >>>> Thomas Koenig wrote:
> >>>>> robf...@gmail.com <robf...@gmail.com> schrieb:
> >>>>>
> >>>>>> 30+ comparators per core (300 comparators?) might allow every
> >>>>>> process to have its own comparator. Which would not need to be
> >>>>>> managed by the OS.
> >>>>>
> >>>>> Hmm... do you want to restrict the core to 300 processes? Hard
> >>>>> limits have a habit of coming around and biting the user when it
> >>>>> least expected (for example when an attacker or a buggy process
> >>>>> does a few forks() and uses up all of them).
> >>>>>
> >>>>> And you would need the OS anyway to schedule access to them so
> >>>>> so it is guaranteed that no two processes use the same timer.
> >>> <
> >>>> You don't need to dedicate a high resolution timer to a process.
> >>>> That wouldn't do much good as processes can have many timers
> >>>> and software within a single process won't know about other timers.
> >>>>
> >>>> Each one-shot HW hr-comparator manages a queue of requests.
> >>> <
> >>> Yes, obviously...
> >>> <
> >>>> The queue can be a linked list or binary tree in ascending order
> >>>> of absolute timeout hr-tick time. The earliest timeout timestamp
> >>>> is written into the trigger register, or 0 to disable the trigger
> >>>> if the queue is empty.
> >>> <
> >>> What does SW do in managing this list of event-times when one
> >>> event fires, and the next event is 1 tick farther down in time ?
> >>> It seems to me that the only way for SW to manage such a queue
> >>> would be to have large timer-quanta.
> >>
> >> I think that there are only two ways it can work:
> >>
> >> a) Make the comparator >= so that when the next value is written, it
> >> immediately causes a timeout since the tick count has passed. This
> >> does require that comparator to separate between a recent rollower and
> >> a long time to go, typically using 2^63 as the boundary:
> >>
> >> diff = next_event_count - current_tick_count;
> >> if ((signed) diff <= 0) fire();
> >
> > Unlike the system clock, the period counter resets to zero at boot.
> > With a 1 ns tick a 64 bit counter wraps after 584 years of
> > continuous up-time so it wouldn't need any extra handling.
> that is a very good, and probably sufficient, reason for making such a
> counter 64-bit only. If you need the ability to work in a 32-bit
> environment, then reading or writing the lower 32 bits would have to
> latch the upper part as well, holding it until the next instruction is done.

If we want reading from user process, which we likely do, then your
suggestion sound dangerous. I don't want to start to think what
happens if task switch hits between two halves of read. May be,
nothing wrong, but proving it causes headaches.

I see nothing particularly wrong with reading pair of registers in loop
hi = ReadHi()
lo = ReadLo()
while ((repHi = readHi()) != hi) {
lo = ReadLo()
hi = repHi
}

It's just a little slower, but painless.

Or, if you insist on reading lower half first:
lo = ReadLo()
hi = ReadHi()
while ((repLo = readLo()) < lo) {
hi = ReadHi()
lo = repLo
}

> >
> > However if the counter or trigger is smaller, say 32 or 48 bits,
> > or the ticks smaller, say 1 ps, then the counter wrap generates
> > an interrupt that increments an epoch number in memory.
> >
> > The trigger register for the youngest timer at the head of the queue
> > is only set when the current epoch number >= that timers' epoch.
> > Otherwise the trigger is disabled.
> OK.
> >
> >> b) Do the same in SW with a polled approach:
> >>
> >> When a event count triggers, the handler dispatches it, then grabs the
> >> next count in the queue and inserts it before immediately reading the
> >> current count to check if the time has already passed, if so, handle
> >> it and repeat. The big problem here is of course that there will
> >> always be a tiny window between writing the next comparator value and
> >> reading the current count to check for an overrun. Fixing that adds a
> >> lot of hair, you either need the (HW) ability to get the current count
> >> at the same time as you write the comparator, or some kind of status
> >> register to report on any currently pending timeout.
> >>
> >> It is very easy to see why the hw people would prefer a plain ==
> >> comparator, but it does lead to even more issues in software. :-(
> >
> > The original spec also had design options like whether the counter
> > or comparators were 32 or 64 bits, and if 64 bits are they read and
> > written atomically or as two separate 32 bit values.
> > Software had to take all these variations into account.
> >
> > One of the interactions the Linux folks seemed concerned
> > about is that an interrupt is generated when the counter wraps,
> > and this can interact with power state management.
> > A 32 bit counter with 1 ns resolution tick would overflow every
> > 4 seconds and wake up the processor out of power down.
> > Another reason to require 64 bit timer components.
> I agree 100%.
> Terje
>
> --
> - <Terje.Mathisen at tmsw.no>
> "almost all programming can be viewed as an exercise in caching"

Michael S wrote:
> On Friday, August 19, 2022 at 10:25:13 AM UTC+3, Terje Mathisen wrote:
>> EricP wrote:
>>> Terje Mathisen wrote:
>>>> MitchAlsup wrote:
>>>>> On Wednesday, August 17, 2022 at 9:14:55 AM UTC-5, EricP wrote:
>>>>>> Thomas Koenig wrote:
>>>>>>> robf...@gmail.com <robf...@gmail.com> schrieb:
>>>>>>>
>>>>>>>> 30+ comparators per core (300 comparators?) might allow every
>>>>>>>> process to have its own comparator. Which would not need to be
>>>>>>>> managed by the OS.
>>>>>>>
>>>>>>> Hmm... do you want to restrict the core to 300 processes? Hard
>>>>>>> limits have a habit of coming around and biting the user when it
>>>>>>> least expected (for example when an attacker or a buggy process
>>>>>>> does a few forks() and uses up all of them).
>>>>>>>
>>>>>>> And you would need the OS anyway to schedule access to them so
>>>>>>> so it is guaranteed that no two processes use the same timer.
>>>>> <
>>>>>> You don't need to dedicate a high resolution timer to a process.
>>>>>> That wouldn't do much good as processes can have many timers
>>>>>> and software within a single process won't know about other timers.
>>>>>>
>>>>>> Each one-shot HW hr-comparator manages a queue of requests.
>>>>> <
>>>>> Yes, obviously...
>>>>> <
>>>>>> The queue can be a linked list or binary tree in ascending order
>>>>>> of absolute timeout hr-tick time. The earliest timeout timestamp
>>>>>> is written into the trigger register, or 0 to disable the trigger
>>>>>> if the queue is empty.
>>>>> <
>>>>> What does SW do in managing this list of event-times when one
>>>>> event fires, and the next event is 1 tick farther down in time ?
>>>>> It seems to me that the only way for SW to manage such a queue
>>>>> would be to have large timer-quanta.
>>>>
>>>> I think that there are only two ways it can work:
>>>>
>>>> a) Make the comparator >= so that when the next value is written, it
>>>> immediately causes a timeout since the tick count has passed. This
>>>> does require that comparator to separate between a recent rollower and
>>>> a long time to go, typically using 2^63 as the boundary:
>>>>
>>>> diff = next_event_count - current_tick_count;
>>>> if ((signed) diff <= 0) fire();
>>>
>>> Unlike the system clock, the period counter resets to zero at boot.
>>> With a 1 ns tick a 64 bit counter wraps after 584 years of
>>> continuous up-time so it wouldn't need any extra handling.
>> that is a very good, and probably sufficient, reason for making such a
>> counter 64-bit only. If you need the ability to work in a 32-bit
>> environment, then reading or writing the lower 32 bits would have to
>> latch the upper part as well, holding it until the next instruction is done.
>
> If we want reading from user process, which we likely do, then your
> suggestion sound dangerous. I don't want to start to think what
> happens if task switch hits between two halves of read. May be,
> nothing wrong, but proving it causes headaches.
>
> I see nothing particularly wrong with reading pair of registers in loop
> hi = ReadHi()
> lo = ReadLo()
> while ((repHi = readHi()) != hi) {
> lo = ReadLo()
> hi = repHi
> }
>
> It's just a little slower, but painless.
>
> Or, if you insist on reading lower half first:
> lo = ReadLo()
> hi = ReadHi()
> while ((repLo = readLo()) < lo) {
> hi = ReadHi()
> lo = repLo
> }

We've discussed this a few years ago, you can even do it when the HW
always require you to read both halves, i.e.

prelo = ReadLo();
prehi = ReadHi();
do {
lo = ReadLo();
hi = ReadHi();
} while (hi != prehi);

You suffer the latency of two full reads and then you have odds of
approx 2^-32 of having to repeat the loop one more time.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

On Friday, August 19, 2022 at 12:47:40 AM UTC+3, MitchAlsup wrote:
> On Thursday, August 18, 2022 at 1:49:27 PM UTC-5, EricP wrote:
> > Terje Mathisen wrote:
> > >
> > > diff = next_event_count - current_tick_count;
> > > if ((signed) diff <= 0) fire();
> > Unlike the system clock, the period counter resets to zero at boot.
> <
> > With a 1 ns tick a 64 bit counter wraps after 584 years of
> > continuous up-time so it wouldn't need any extra handling.
> <
> At least the person who turned the system on won't have to listen
> to the complaints of bad-SW when the timer does roll over.
> >
> > However if the counter or trigger is smaller, say 32 or 48 bits,
> > or the ticks smaller, say 1 ps, then the counter wrap generates
> > an interrupt that increments an epoch number in memory.
> <
> So, in effect, you are arguing that the timer resolution should
> be better than 1ns and worse than 1ps. 1ps rolls a 64-bit counter
> over every ½ year, 10ps every 5.8 years, 100 ps every 58 years.
> >
> > >
> > The original spec also had design options like whether the counter
> > or comparators were 32 or 64 bits, and if 64 bits are they read and
> > written atomically or as two separate 32 bit values.
> <
> In this day and age, why would anyone want something smaller than
> a 64-bit counter ???
> <
> > Software had to take all these variations into account.
> >
> > One of the interactions the Linux folks seemed concerned
> > about is that an interrupt is generated when the counter wraps,
> > and this can interact with power state management.
> > A 32 bit counter with 1 ns resolution tick would overflow every
> > 4 seconds and wake up the processor out of power down.
> > Another reason to require 64 bit timer components.
> <
> I repeat::
> In this day and age, why would anyone want something smaller than
> a 64-bit counter ???

That's a good question :(

As a matter of fact, a 32-bit MCU that I was working on in recent weeks
(fixing obscure bug left here by former employee) has 14 timers. Only
2 of them are 32-bit. The rest is 16 bit.

Another bit of trivia - in ARMv7M architecture SysTick timer is 24-bit.

On Friday, August 19, 2022 at 1:37:38 PM UTC+3, Terje Mathisen wrote:
> Michael S wrote:
> > On Friday, August 19, 2022 at 10:25:13 AM UTC+3, Terje Mathisen wrote:
> >> EricP wrote:
> >>> Terje Mathisen wrote:
> >>>> MitchAlsup wrote:
> >>>>> On Wednesday, August 17, 2022 at 9:14:55 AM UTC-5, EricP wrote:
> >>>>>> Thomas Koenig wrote:
> >>>>>>> robf...@gmail.com <robf...@gmail.com> schrieb:
> >>>>>>>
> >>>>>>>> 30+ comparators per core (300 comparators?) might allow every
> >>>>>>>> process to have its own comparator. Which would not need to be
> >>>>>>>> managed by the OS.
> >>>>>>>
> >>>>>>> Hmm... do you want to restrict the core to 300 processes? Hard
> >>>>>>> limits have a habit of coming around and biting the user when it
> >>>>>>> least expected (for example when an attacker or a buggy process
> >>>>>>> does a few forks() and uses up all of them).
> >>>>>>>
> >>>>>>> And you would need the OS anyway to schedule access to them so
> >>>>>>> so it is guaranteed that no two processes use the same timer.
> >>>>> <
> >>>>>> You don't need to dedicate a high resolution timer to a process.
> >>>>>> That wouldn't do much good as processes can have many timers
> >>>>>> and software within a single process won't know about other timers.
> >>>>>>
> >>>>>> Each one-shot HW hr-comparator manages a queue of requests.
> >>>>> <
> >>>>> Yes, obviously...
> >>>>> <
> >>>>>> The queue can be a linked list or binary tree in ascending order
> >>>>>> of absolute timeout hr-tick time. The earliest timeout timestamp
> >>>>>> is written into the trigger register, or 0 to disable the trigger
> >>>>>> if the queue is empty.
> >>>>> <
> >>>>> What does SW do in managing this list of event-times when one
> >>>>> event fires, and the next event is 1 tick farther down in time ?
> >>>>> It seems to me that the only way for SW to manage such a queue
> >>>>> would be to have large timer-quanta.
> >>>>
> >>>> I think that there are only two ways it can work:
> >>>>
> >>>> a) Make the comparator >= so that when the next value is written, it
> >>>> immediately causes a timeout since the tick count has passed. This
> >>>> does require that comparator to separate between a recent rollower and
> >>>> a long time to go, typically using 2^63 as the boundary:
> >>>>
> >>>> diff = next_event_count - current_tick_count;
> >>>> if ((signed) diff <= 0) fire();
> >>>
> >>> Unlike the system clock, the period counter resets to zero at boot.
> >>> With a 1 ns tick a 64 bit counter wraps after 584 years of
> >>> continuous up-time so it wouldn't need any extra handling.
> >> that is a very good, and probably sufficient, reason for making such a
> >> counter 64-bit only. If you need the ability to work in a 32-bit
> >> environment, then reading or writing the lower 32 bits would have to
> >> latch the upper part as well, holding it until the next instruction is done.
> >
> > If we want reading from user process, which we likely do, then your
> > suggestion sound dangerous. I don't want to start to think what
> > happens if task switch hits between two halves of read. May be,
> > nothing wrong, but proving it causes headaches.
> >
> > I see nothing particularly wrong with reading pair of registers in loop
> > hi = ReadHi()
> > lo = ReadLo()
> > while ((repHi = readHi()) != hi) {
> > lo = ReadLo()
> > hi = repHi
> > }
> >
> > It's just a little slower, but painless.
> >
> > Or, if you insist on reading lower half first:
> > lo = ReadLo()
> > hi = ReadHi()
> > while ((repLo = readLo()) < lo) {
> > hi = ReadHi()
> > lo = repLo
> > }
> We've discussed this a few years ago, you can even do it when the HW
> always require you to read both halves, i.e.
>
> prelo = ReadLo();
> prehi = ReadHi();
> do {
> lo = ReadLo();
> hi = ReadHi();
> } while (hi != prehi);
>
> You suffer the latency of two full reads and then you have odds of
> approx 2^-32 of having to repeat the loop one more time.

So, don't build HW that requires reading of both halves!
It's a bad idea for few other reasons except latency of the 2nd read.

If you ask me, don't build HW with non-idempotent reads. Period.

BTW, the code above does not work. It lacks update of prehi.

> Terje
>
> --
> - <Terje.Mathisen at tmsw.no>
> "almost all programming can be viewed as an exercise in caching"

Michael S wrote:
> On Friday, August 19, 2022 at 1:37:38 PM UTC+3, Terje Mathisen wrote:
>> Michael S wrote:
>>> I see nothing particularly wrong with reading pair of registers in loop
>>> hi = ReadHi()
>>> lo = ReadLo()
>>> while ((repHi = readHi()) != hi) {
>>> lo = ReadLo()
>>> hi = repHi
>>> }
>>>
>>> It's just a little slower, but painless.
>>>
>>> Or, if you insist on reading lower half first:
>>> lo = ReadLo()
>>> hi = ReadHi()
>>> while ((repLo = readLo()) < lo) {
>>> hi = ReadHi()
>>> lo = repLo
>>> }
>> We've discussed this a few years ago, you can even do it when the HW
>> always require you to read both halves, i.e.
>>
>> prelo = ReadLo();
>> prehi = ReadHi();
>> do {
>> lo = ReadLo();
>> hi = ReadHi();
>> } while (hi != prehi);
>>
>> You suffer the latency of two full reads and then you have odds of
>> approx 2^-32 of having to repeat the loop one more time.
>
> So, don't build HW that requires reading of both halves!
> It's a bad idea for few other reasons except latency of the 2nd read.
>
> If you ask me, don't build HW with non-idempotent reads. Period.

Strong YES!

>
> BTW, the code above does not work. It lacks update of prehi.

Yeah, I realized that shortly after posting but was going to allow you
to point it out. :-)

lo = ReadLo();
hi = ReadHi();
do {
prelo = lo; prehi = hi;
lo = ReadLo();
hi = ReadHi();
} while (hi != prehi);

is much cleaner looking as well.

If I was forced to work with such a timer I would still prefer the
interface to be 64-bit, i.e. hide the ugly re-read loop inside a system
library:

uint64_t readCounter()
{ uint64_t r;
uint32_t lo, hi, prelo, prehi;
lo = ReadLo(); hi = ReadHi();
do {
prelo = lo; prehi = hi;
lo = ReadLo();
hi = ReadHi();
r = hi;
} while (hi != prehi);
r = (r << 32) | lo;
return r;
}

This way you can keep the interface when you get a working (64-bit)
hardware interface.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

On Friday, August 19, 2022 at 5:06:10 AM UTC-5, Michael S wrote:
> On Friday, August 19, 2022 at 10:25:13 AM UTC+3, Terje Mathisen wrote:

> I see nothing particularly wrong with reading pair of registers in loop
> hi = ReadHi()
// context switch happens here
> lo = ReadLo()
> while ((repHi = readHi()) != hi) {
> lo = ReadLo()
// context switch happens here
> hi = repHi
> }
>
> It's just a little slower, but painless.
>
> Or, if you insist on reading lower half first:
> lo = ReadLo()
// context switch happens here
> hi = ReadHi()
> while ((repLo = readLo()) < lo) {
> hi = ReadHi()
// context switch happens here
> lo = repLo
> }
>

In my test system there are two timer groups. Each group contains up to
32 timers requiring a 2kB address range. Should the timer groups be present
in the same MMU page, or should they have separate pages allocated for
each group? Using separate pages the two groups can be isolated so that
only the group being accessed is mapped and alterable. However, it may be
simpler just to map all the timers using one MMU page. It is basically the
same device broken into groups for hardware convenience.

On Monday, February 27, 2023 at 5:33:31 PM UTC-6, robf...@gmail.com wrote:
> In my test system there are two timer groups. Each group contains up to
> 32 timers requiring a 2kB address range.
<
32 sixty-four (64-bit) counters are only 256 bytes. What kind of data resides
between the counters; how big are these counters ??
<
> Should the timer groups be present
> in the same MMU page, or should they have separate pages allocated for
> each group?
<
The OS-counter-driver writer should make this decision.
<
> Using separate pages the two groups can be isolated so that
> only the group being accessed is mapped and alterable. However, it may be
> simpler just to map all the timers using one MMU page. It is basically the
> same device broken into groups for hardware convenience.
<
My context switch drags around 8 counters (on a per thread basis).
<
What are these counter/timers being used for ??

"robf...@gmail.com" <robfi680@gmail.com> writes:
>In my test system there are two timer groups. Each group contains up to
>32 timers requiring a 2kB address range. Should the timer groups be present
>in the same MMU page, or should they have separate pages allocated for
>each group? Using separate pages the two groups can be isolated so that
>only the group being accessed is mapped and alterable. However, it may be
>simpler just to map all the timers using one MMU page. It is basically the
>same device broken into groups for hardware convenience.

If you want to give the timers to guest operating systems, then they
need to be spaced by the size of the largest page.

Or programmable via a PCI-like BAR.

On Monday, February 27, 2023 at 7:20:32 PM UTC-5, Scott Lurndal wrote:
> "robf...@gmail.com" <robf...@gmail.com> writes:
> >In my test system there are two timer groups. Each group contains up to
> >32 timers requiring a 2kB address range. Should the timer groups be present
> >in the same MMU page, or should they have separate pages allocated for
> >each group? Using separate pages the two groups can be isolated so that
> >only the group being accessed is mapped and alterable. However, it may be
> >simpler just to map all the timers using one MMU page. It is basically the
> >same device broken into groups for hardware convenience.
> If you want to give the timers to guest operating systems, then they
> need to be spaced by the size of the largest page.
>
> Or programmable via a PCI-like BAR.

Each group has a PCI config space.
They have programmable addresses via PCI config BAR.

>32 sixty-four (64-bit) counters are only 256 bytes. What kind of data resides
>between the counters; how big are these counters ??

I have changed this to 64 counters, it is configurable up to 64.
There are four 64-bit registers associated with each counter,
current count, max count, the on time, and a control reg.

There are also eight more registers to control interrupt enable,
gating counters on and off, synchronizing counters and viewing the output
pulse. So the registers occupy a 4kB Area with a lot of unused space.

The counter registers are just holding registers, which do not update the
counter until the sync register is written. This allows all counters to be updated
at the same time.

>What are these counter/timers being used for ??
OS: time slice, garbage collect / watchdog and disk clock algorithm, games and
possibly robotics.

Four of the timers have external ports for gating and output. The 'on' time
controls the pulse width of the timer.

On Monday, February 27, 2023 at 8:01:27 PM UTC-5, robf...@gmail.com wrote:
> On Monday, February 27, 2023 at 7:20:32 PM UTC-5, Scott Lurndal wrote:
> > "robf...@gmail.com" <robf...@gmail.com> writes:
> > >In my test system there are two timer groups. Each group contains up to
> > >32 timers requiring a 2kB address range. Should the timer groups be present
> > >in the same MMU page, or should they have separate pages allocated for
> > >each group? Using separate pages the two groups can be isolated so that
> > >only the group being accessed is mapped and alterable. However, it may be
> > >simpler just to map all the timers using one MMU page. It is basically the
> > >same device broken into groups for hardware convenience.
> > If you want to give the timers to guest operating systems, then they
> > need to be spaced by the size of the largest page.
> >
> > Or programmable via a PCI-like BAR.
> Each group has a PCI config space.
> They have programmable addresses via PCI config BAR.
> >32 sixty-four (64-bit) counters are only 256 bytes. What kind of data resides
> >between the counters; how big are these counters ??
> I have changed this to 64 counters, it is configurable up to 64.
> There are four 64-bit registers associated with each counter,
> current count, max count, the on time, and a control reg.
>
> There are also eight more registers to control interrupt enable,
> gating counters on and off, synchronizing counters and viewing the output
> pulse. So the registers occupy a 4kB Area with a lot of unused space.
>
> The counter registers are just holding registers, which do not update the
> counter until the sync register is written. This allows all counters to be updated
> at the same time.
> >What are these counter/timers being used for ??
> OS: time slice, garbage collect / watchdog and disk clock algorithm, games and
> possibly robotics.
>
> Four of the timers have external ports for gating and output. The 'on' time
> controls the pulse width of the timer.

I forgot, four of the timers can also be clocked by an external clock, so they can
be used to count pulses.